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Executive Summary 



The Toolbox Revisited is a data essay that follows a nationally representative cohort of students 
from high school into postsecondary education, and asks what aspects of their formal schooling 
contribute to completing a bachelor’s degree by their mid-20s. The universe of students is 
confined to those who attended a four-year college at any time, thus including students who 
started out in other types of institutions, particularly community colleges. 

The core question, data source, and legacy 

The core question is not about basic “access” to higher education. It is not about persistence to 
the second term or the second year following postsecondary entry. It is about completion of 
academic credentials — the culmination of opportunity, guidance, choice, effort, and commitment. 

To answer the question, The Toolbox Revisited uses the most recently completed of the national 
grade-cohort longitudinal studies conducted by the National Center for Education Statistics. 

This study, known as the NELS:88/2000, began with a national sample of eighth-graders in 
1988. They were scheduled to be in the 12th grade and graduate from high school in 1992. 

They were followed through December 2000. In addition to regular interviews with these 
students, the data set on which this essay draws includes the critical components of high school 
and college transcripts, and the transcript data are the principal sources for the academic history 
observed. 

The Toolbox Revisited was designed as a replication of a noted previous study published by the 
U.S. Department of Education, Answers in the Tool Box: Academic Intensity, Attendance 
Patterns, and Bachelor’s Degree Attainment (1999), hereinafter referred to as "the original Tool 
Box," which based its analysis on a national cohort of high school students who were scheduled 
to graduate in 1982, and who were followed through 1993. The question naturally arose as to 
whether the hypotheses and analyses based on that cohort’s history would hold up in the story of 
the slightly overlapping 1992-2000 period. 

We have learned a great deal in a very short time from numerous initiatives of states and private 
foundations to prepare high school students better for higher education, and from major federal 
stimuli under the No Child Left Behind legislation to jump start the process of academic 
momentum prior to high school. One of the reasons for examining the academic history of the 
NELS:88/2000 cohort is that its students attended high school after the wave of reforms in the 
1980s that followed the critique of U.S. education offered by the seminal report, A Nation at Risk 
(1983), and, hence, may provide some clues as to the likely outcomes of current reform efforts. 

Much has changed in other ways, too, since the High School Class of 1982 (the subjects of the 
original Tool Box ) moved through their scheduled 12th grade and through postsecondary 
education. A dramatically higher proportion of high school seniors of all race/ethnicity groups 
continue their education, though access gaps remain (Wirt et al. 2005, indicator 22). 
Postsecondary attendance patterns among traditional-age students have become far more 
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complex, with nearly 60 percent of undergraduates attending more than one institution, and 35 
percent of this group crossing state lines in the process; community college transfer rates rising 
nearly 10 percentage points; one out of eight undergraduates based in four-year institutions using 
community colleges to fill in pieces of their curriculum, and another eight percent “swirling” 
back and forth between the four-year and two-year sectors. Dual-enrollment while in high 
school, credit-by-examination, and use of summer tenns all added to the dynamic mix of time 
and space that marked student pathways in the 1990s. 

With all this change, we still measure something called “college graduation rates” with 
anachronistic formulas that do not track students through increasingly complex paths to degrees. 
As a result, we do not understand what is really going on. The dominant language 
accompanying analyses bemoaning putatively low graduation rates is a language of “attrition,” 
with students labeled “at risk” or “minimally college-qualified,” and leaking out of “pipelines.” 

This study looks at student histories derived from transcript records in a different way and with a 
different tone. It follows the student, not the institution, because it is the student’s success that 
matters to families — and to the nation. It allows the maximum length of postsecondary time for 
the High School Class of 1992, 8.5 years, for students to earn degrees no matter how many 
institutions they attend. It notes that if the history of the Class of 1982 were truncated at 8.5 
years, there has been a decent improvement in bachelor’s degree attainment among non- 
incidental students (those who earned more than 10 credits, i.e. "made a go of it") who attended a 
four-year college at any time (from 60 to 66 percent). It is natural to ask how this happened, to 
identify the moments and aspects of schooling that may have made a difference, and to reflect on 
what might make the most difference in the future for narrowing degree-completion gaps by 
race/ethnicity. In that task, The Toolbox Revisited looks for the features of academic history that 
are realistically subject to change by institutions whose principal business is the generation, 
preservation, and dissemination of knowledge. While acknowledging that for degree-completion 
rates to improve, students themselves must respond, and that their response does not occur in a 
vacuum, the features of student histories that are the domain of this inquiry do not include social 
and psychological variables attendant on the passage from adolescence to adulthood. 

It is important to note that, as was the case for the original Tool Box, the student universe for 
The Toolbox Revisited constitutes roughly half who reach the 12th grade (table 1). It does not 
include students who failed to graduate from high school, those who earned General Education 
Diplomas (GEDs), those who had not enrolled in any postsecondary institution by the age of 26, 
and those who entered the postsecondary system but never attended a bachelor’s degree-granting 
institution. The resulting demographics are slightly more female, slightly less minority, less with 
a second language background, and a higher socioeconomic status distribution than the cohort as 
a whole (table 2). 

Organization 

We learned from critiques of the original Tool Box to sort the chronology of events with greater 
care. There are seven steps in the analysis of The Toolbox Revisited, each of which involves a 



-xvi- 




collection of variables that are investigated in terms of the degree to which they help us explain 
bachelor’s degree completion for the population of students under investigation: 

Step 1 : Demographic background and high school history 
Step 2: Postsecondary entrance (timing and type of institution) 

Step 3: First postsecondary year history (curriculum and performance) 

Step 4: Factors of financing postsecondary education in the early years 
Step 5: Postsecondary attendance patterns 

Step 6: Extended postsecondary history (curriculum and perfonnance) 

Step 7: Final model, with complete academic history 

This essay also takes an important pause outside the steps of the core statistical model to 
consider the characteristics of student progress through the second year following postsecondary 
entry. 

As each step of the statistical model of student history is set forth, it is pointed out where the 
results are similar to the findings of the original Tool Box with its earlier population, and where 
they diverge. All seven steps are subsequently assembled together in one place (table 29) so that 
the reader can observe the factors that have consistently contributed to degree completion. 

Principal Stories and Guidance 

This executive summary offers themes, highlights, and implications of the data for those who 
comment on secondary and higher education and make decisions about institutional or system 
policy — editorial writers, legislators, researchers, education administrators. 

Two national longitudinal studies, a decade apart, have told similar stories. When the second 
story reinforces the first — and sheds even more light — something has to be right, and it behooves 
us to pay attention. Both of them provide support for current efforts to improve the quality of 
high school curricula and the participation in those curricula of ever larger proportions of 
students. Both of them provide guidance for college and community college processes likely to 
lead students to degree completion. 

Some of what was learned from the original Tool Box was taken to heart at the secondary school 
level, and, in some respects, we are seeing positive results in academic curricular participation in 
high schools. But counting Carnegie units' in English or science is not the same as describing 
and validating what students have learned, and whether that learning links smoothly to the 
performance expectations of the postsecondary world. The Toolbox Revisited says we have more 
to do, that the bulk of our task lies both after the college matriculation line, and in 
communication and outreach between postsecondary institutions and high schools. How do we 



*A Carnegie unit is the basic credit system for U.S. secondary schools. It is generally recognized as 
representing a full year (36-40 weeks) in a specific class meeting four or five times per week for 40-50 minutes per 
class session (Martinez and Bray 2002). 
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learn what we have to do? By following students in the richness and complexity of their 
postsecondary histories. 

Curriculum, starting in high school, and continuing 

However complex students’ attendance patterns, the principal story line leading to degrees is that 
of content. What one learns is what one studies, and what one brings to economic and 
community life. The story starts in high school, but merely crossing the bridge to college or 
community college doesn’t mean the story is over. Furthermore, the bridge is not always aligned 
with the road on the other side. 

The academic intensity of the student’s high school curriculum still counts more than anything 
else in precollegiate history in providing momentum toward completing a bachelor’s degree. At 
the highest level of a 31 -level scale describing this academic intensity (see Appendix F), one 
finds students who, through grade 12 inl992, had accumulated: 

3.75 or more Carnegie units of English 
3.75 or more Carnegie units of mathematics 

highest mathematics of either calculus, precalculus, or trigonometry 
2.5 or more Carnegie units of science or more than 2.0 Carnegie units of core 
laboratory science (biology, chemistry, and physics) 
more than 2.0 Carnegie Units of foreign languages 
more than 2.0 Carnegie Units of history and social studies 
1 .0 or more Carnegie Units of computer science 
more than one Advanced Placement course 
no remedial English; no remedial mathematics 

These are minimums. In fact, students who reached this level of academic curriculum intensity 
accumulated much more than these threshold criteria (see table FI), and 95 percent of these 
students earned bachelor’s degrees (41 also percent earned master’s, first professional, or 
doctoral degrees) by December 2000. 

Provided that high schools offer these courses, students are encouraged or required to take them, 
and, in the case of electives, students choose to take them, just about everybody could 
accumulate this portfolio. Unfortunately, not all high schools present adequate opportunity-to- 
learn, and some groups of students are excluded more than others. Latino students, for example, 
are far less likely to attend high schools offering trigonometry (let alone calculus) than white or 
Asian students. Students from the lowest socioeconomic status (SES) quintile attend high 
schools that are much less likely to offer any math above Algebra 2 than students in the upper 
SES quintiles (table 6). If we are going to close gaps in preparation — and ultimate degree 
attainment — the provision of curriculum issue has to be addressed. In recent years, colleges and 
community colleges have begun to provide these courses to high school students, and distance 
learning provides additional options if students have access to the technology. The hypothetical 




consequences of participating in curriculum configurations approaching that illustrated above for 
Latino degree completion rates, in particular, are stunning (table 32). 

There is a quantitative theme to the curriculum story that illustrates how students cross the 
bridge onto and through the postsecondary landscape successfully. The highest level of 
mathematics reached in high school continues to be a key marker in precollegiate momentum, 
with the tipping point of momentum toward a bachelor’s degree now firmly above Algebra 2. 

But in order for that momentum to pay off, earning credits in truly college-level mathematics on 
the postsecondary side is de rigeur. The world has gone quantitative: business, geography, 
criminal justice, history, allied health fields — a full range of disciplines and job tasks tells 
students why math requirements are not just some abstract school exercise. By the end of the 
second calendar year of enrollment, the gap in credit generation in college-level mathematics 
between those who eventually earned bachelor’s degrees and those who didn’t is 71 to 38 
percent (table 21). In a previous study, the author found the same magnitude of disparity among 
community college students in relation to earning a terminal associate degree (Adehnan 2005a). 
The math gap is something we definitely have to fix. 

A dominant feature of academic histories that cannot really be assessed until the end of the 
second year following college entry is the extent to which students successfully completed 
credits in a range of “gateway” courses. It is at this point that the postsecondary curricular story 
line fully emerges, with ratios of participation in the “gateways” between those who ultimately 
earned degrees and those who did not running 6: 1 in American literature, 4: 1 in general 
chemistry, and more than 3: 1 in precalculus, micro/macroeconomics, introduction to philosophy, 
and world civilization (table 20). These gaps in curricular participation argue for academic 
administrators to identify their key gateway courses and regularly monitor participation. 

College and community college expectations for their first-year students in those gateway 
courses — expressed through examinations, paper and laboratory assignments — need to be more 
public. Examples such as those offered by the American Diploma Project in its report, Ready or 
Not: Creating a High School Diploma That Counts (2004), should be shared with larger 
audiences than policymakers and others who habitually read such reports. Parents should see 
those assignments even if they don’t understand them; high school teachers should ponder them 
to assess whether their exiting students are likely to be prepared; and, most importantly, high 
school students have got to see them as road signs to their next education destination. The Tool- 
box Revisited advocates making these examples part and parcel of admissions packets, publicity 
brochures, and Web sites. There is risk in this: Some students may be scared away. But there is 
no better way to enhance articulation and preparedness than to display what students can expect. 

Postsecondary benchmarks 

In both colleges and community colleges, the curriculum story line intersects attendance patterns 
and performance in ways that set benchmarks for academic advisement and intervention: 
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• Less than 20 credits by the end of the first calendar year of enrollment (no matter 
in what term one started, whether summer, fall, winter, spring) is a serious drag 
on degree completion. The original Tool Box told the same story. It is all the 
more reason to begin the transition process in high school with expanded dual 
enrollment programs offering true postsecondary course work so that students 
enter higher education with a minimum of 6 additive credits to help them cross 
that 20-credit line. Six is good, 9 is better, and 12 is a guarantee of momentum. 

• We falsely believe that beginning students drop out of higher education in 
appalling numbers by the end of their scheduled first academic year of attendance. 
In fact, about 90 percent of traditional-age beginning students turn up somewhere 
(maybe not at the first school attended) and at some time (maybe not in the fall 
tenn) during the subsequent calendar academic year (which we measure as July 1 
through June 30). However impressive this percentage, the quality of persistence 
counts more, and, for a third of these students, the quality of persistence leaves 
much to be desired (table 17). The Toolbox Revisited urges that institutions 
monitor and report the quality (as much as the fact) of persistence. 

• More than 60 percent of the students in the sample under investigation enrolled 
during summer terms. Undergraduates are not only more geographically mobile, 
but have shattered observance of the traditional academic calendar. Summer term 
credits are more than metaphors for high octane persistence: Earning more than 

4 credits during those terms held a consistently positive relationship to degree 
completion, and gave African-American students, in particular, a significant boost 
in hypothetical graduation rates (table 32). College and community college 
administrators can be very creative in expanding the use of summer tenns. 

Student uses of time 

The example of summer-term credits, particularly in combination with the complex multi- 
institutional attendance patterns, underscores another theme of The Toolbox Revisited : Student 
uses of time in undergraduate careers are now more important than their uses of place. In other 
words, when students do something academic has a more significant relationship to degree 
completion than where they do it. For example: 

• For the High School Class of 1982 (the subjects of the original Tool Box), timing 
of entry to postsecondary education never rose to a level of statistical significance 
in the analysis, whereas variables for the type of institution first entered played 
inconsistent but positive roles in explaining degree completion. A decade later, 
with a higher proportion of high school students continuing to college, the 
situation was reversed (table 13). What this means is that recruitment efforts have 
to insure that students enter postsecondary education immediately following high 
school graduation. The longer students wait, the less likely they will finish a 
degree. 
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• The only characteristic of the first institution of attendance to be admitted to 
statistical analysis was selectivity, but it never rose above the threshold of 
significance. Quite frankly, one isn’t worried about degree completion for the 5 
percent of traditional-age undergraduates who enter highly selective colleges. 

One is more concerned with the rest of the river — particularly the 78 percent who 
start in either nonselective four-year colleges or open-door community colleges. 

• The original Tool Box study declined to confront part-time status and its effects. 

If one is using transcripts as evidence, there are a number of problems in 
determining which students are part-time and when. The Toolbox Revisited found 
a way around these problems to mark whether a student’s enrollment intensity 
ever fell into part-time status, i.e., less than 12 credits per semester or its 
equivalent. Part-time attendance by whatever means, as Carroll (1989) labeled it, 
proved “hazardous” to degree completion health (table 24; table 29). 

• In longitudinal studies extending for as long a period of postsecondary time as 
does the NELS:88/2000 (8.5 calendar years), a student is allowed stop-out periods 
totaling one semester or its equivalent (e.g., two quarters), exclusive of summer 
tenns, and still be considered "continuously enrolled." Continuous enrollment is 
a factor of attendance patterns, and another marker of the student’s use of time. It 
proves to be overpowering: with 16 other variables in play, continuous enrollment 
increases the probability of degree completion by 43 percent (table 27). The 
original Tool Box offered the same message, arguing for assiduous monitoring of 
student stop-out periods. Put another way: Keep the student continuously 
enrolled, even part-time (less damaging than excessive stop-out periods). 

Purposeful migration versus "swirling" 

The complexity of student postsecondary enrollment patterns, already a notable phenomenon for 
the population under study in the original Tool Box, accelerated in the subsequent cohort. The 
construction of the NELS:88/2000 postsecondary transcript files took advantage of what we 
learned from more sophisticated institutional and state system tracking studies of the 1990s; 
hence, some new attendance pattern variables were available and others (those describing 
different kinds of multi-institutional attendance) refined. 

What we found for the students of the 1992-2000 period was this: 

• Formal transfer from a community college to a four-year college and formal 
transfer from one four-year college to another were positively associated with 
degree completion, but wandering from one school to another was not. 

In fact, the nomadic multi-institutional attendance behavior increasingly known as ‘swirling,’ 
held a significant and negative relationship to degree completion (table 24, table 39). These 
statements are a very simple untangling of complex realities. 
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The basic question asked of the transcript data — did a student attend only one school or more 
than one? — begins a process of inquiry to determine how the student attended second and third 
institutions. Given very taut definitions of what transfer means, we are advised to ensure that 
multi-institutional attendance is purposeful and productive. For that, we require much better 
student tracking systems than we currently possess, and regular contact with students in motion. 

Student academic performance 

More than the original Tool Box, The Toolbox Revisited recognizes that the path of student 
academic performance, marked by grades, is a reflection of quality of effort, and pays off. It 
starts in high school: Academic curriculum participation is still the strongest of the precollegiate 
momentum indicators, but between the 1980s and 1990s, class rank/GPA moved markedly ahead 
of senior year test score in its contribution to students’ overall "Academic Resources" index, a 
composite indicator of high school curriculum intensity, class rank/GPA, and senior year scores 
on a 90-minute exam best described as a mini, enhanced SAT (see p. 16 and Glossary). 

This story continues on the postsecondary side of the matriculation line: 

• Earning grades that place one in the top 40 percent of first-year GPA for the 
whole cohort is a strong — and positive — contributor to academic momentum, and 
remains in the account of degree completion throughout the histories of both the 
class of 1982 and the class of 1992 (table 15). 

• The theme of quality-of-student-effort, reflected in grades, is strengthened when 
the canvas covers the student’s entire undergraduate career. In the original Tool 
Box, the variable describing the trend in students’ GPA had only two reference 
points: first calendar year and final GPA. For The Toolbox Revisited, there are 
three such points: first calendar year GPA, cumulative GPA for the first two 
calendar years, and GPA as of the last date of attendance, whether or not a degree 
was earned. A rising trend in grades fits with attainment (table 25), contributing 
positively and significantly (table 26). 

A story twice told should be a story to which we listen 

Both the original Tool Box and The Toolbox Revisited revealed that one of the most degree- 
crippling features of undergraduate histories is an excessive volume of courses from which the 
student withdrew without penalty and those the student repeated. We set this up as a ratio, and 
marked those who withdrew from or repeated 20 percent or more of their course attempts. 

Doing so cuts the probability of completing a degree in half (table 27)! 

The withdrawals counted here are not “drop” grades that apply during standard drop-and-add 
periods at the beginning of terms. They are the result of institutional policies that allow 
withdrawals without penalty after the drop-and-add period. No-credit repeats are standard fare 
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in remedial courses, but when they reach destructive levels the question arises as to how many 
times an institution allows a student to repeat a course. Think of it this way: Every non-penalty 
withdrawal and no-credit repeat means that a seat in a course is not available to someone else. 
Add those seats up, and admission to an institution may not be available to someone else. 
Excessively lax withdrawal and repeat policy, then, ultimately blocks general access. And in 
tenns of degree completion, such policies do students no favors. 

What Does Not Count in The Account of Completion? 

• Students’ education “anticipations” (the consistency and level of their vision of 
how far they will get in school) were not significant at any step of the logistic 
account for the High School Class of 1992. This is a change from the position of 
the “anticipations” variable in the original Tool Box, where it ducked in and out of 
significance. The new message is more clear: Among students who attend a four- 
year college at some time, expectations are distinctly secondary to one’s uses of 
academic time and to one’s academic performance. 

• Whereas grants and student work-study were modestly significant contributors to 
degree momentum at early stages of students’ postsecondary careers in the history 
of the High School Class of 1982, the data on finance mechanisms for the High 
School Class of 1992 are poor, and the results inconclusive. Analysts are directed 
instead to the Beginning Postsecondary Students longitudinal studies, which 
contain detailed financial aid data (but skeletal infonnation on high school 
histories and postsecondary course work). 

• Of student demographic characteristics, only one — socioeconomic status — was 
significantly associated with degree completion, though in a modest manner. 
Gender and race/ethnicity were never significant in the logistic narrative, even 
though some indirect effects of these key demographic characteristics would 
probably be found in other statistical models. When each race/ethnicity group 
was treated as an independent variable, the basic story did not change. 

• Both a dichotomous variable marking any remedial work in the first calendar year 
of attendance, and an elaborate variable describing types and extent of 
remediation over the course of a student’s entire undergraduate career were 
employed in the analysis, but to no avail. The same procedure was used in the 
original Tool Box, where the variables were admitted to the statistical model but 
did not reach the threshold of significance. Sufficient numbers of students who 
took remedial classes successfully moved through them so that remediation did 
not make a strategic difference in degree completion. 

• Half of the students in the sample for The Toolbox Revisited who earned 
bachelor’s degrees changed their major along the way. It was natural to ask 
whether change-of-major had any influence on degree attainment. It did not, 
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principally because, with few exceptions, community college transfer students 
come in to the four-year institution from a general studies program and 
automatically are classified as “change-of-major” the minute they enter a specific 
program at the four-year school. 

Students as active, responsible participants 

The Toolbox Revisited does not treat students as passive creatures whose fate is wholly molded 
by schools and colleges. It demonstrates that, within the population of traditional-age students 
who attend a four-year college at any time (obviously including community college transfers), 
we can improve graduation rates and close some of the gaps in completion by race/ethnicity and 
socioeconomic status. But it also argues that there is a limit to what we can realistically do 
unless students respond to highly targeted advice and prodding. 

The analysis of The Toolbox Revisited identifies features of academic history that are most 
tractable in terms of second party intervention. But there is also something we might dub “first 
party intervention.” Once the modest echoes of socioeconomic status are accounted for, each 
step of academic history offers students a set of decisions that require the commitment of time 
and effort likely to yield the return of earning a degree. Provided there is opportunity, the 
choices made by students, beginning with high school curriculum and quality of effort in high 
school, allow subsequent leverage. Entering a postsecondary institution directly from high 
school, earning 20 or more credits in the first calendar year of enrollment, and perfonning well 
enough in that first calendar year to fall in the top 40 percent of a GPA distribution build on 
previous academic investments, and are all signs of commitment. 

Subsequent choices that may not be reflected in a bounded period of time, such as excessive 
course withdrawals, prove to be poor decisions with negative returns, breaking accumulated 
momentum. Other configurations of choice, including summer-term credit generation, meeting 
the challenge of college-level mathematics, effort required to yield a rising GPA, and most of all, 
remaining continuously enrolled, all reflect continuing leverage of attainment. This is what 
academic momentum is all about. While these choices do not take place in a social and 
psychological vacuum, this is a story about the intersection of student choice with the structures 
of opportunity offered by institutions whose first order of business is the distribution of 
knowledge. It is not a story about growing up, although that happens along the way. 

Degree Completion: How High Can We Go? How Much Can the Gaps Be Closed? 

In Part V of The Toolbox Revisited, three different national longitudinal studies conducted during 
the 1990s are set side-by-side, so as to demonstrate a remarkable degree of agreement on the rate 
of bachelor’s degree completion for students who started out in four-year colleges (granted, that 
is only part of the broader universe addressed in this essay). Looking at the concordance of these 
three sources (table 30), it is fair to say that: 

• A third of traditional-age students who started in a four-year college earned a 
bachelor’s degree from the same school in the "traditional" four-year period. 
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• Between 54 and 58 percent earned the degree from the same school in which they 
began within six years of entry. 

• When the option of earning a degree from a different four-year college than the 
one in which these students commenced study, the six-year completion rates are 
in the 62-67 percent range. 

• Only the NELS:88/2000 extends the time period for earning a degree beyond six 
years; at 8.5 years, its degree completion rate for students who started in a four- 
year college approaches 70 percent. 

However, it is unfortunate to note that despite increased participation of minority students to 
postsecondary education over the past quarter century, the gap in bachelor’s degree completion 
between whites and Asians, on the one hand, and Latinos and African-Americans, on the other, 
remains wide. 

What features of academic history might close the gaps, and by how much? 

The data-driven exercise in Part V of The Toolbox Revisited can be characterized as "reasoned 
speculation." From the NELS:88/2000, we start with a degree completion gap between whites 
and Asians vis-a-vis African-Americans of 15 percent; and with reference to Latinos, 22 percent. 
We go back through our analysis and ask what factors: 

(a) consistently contributed to bachelor’s degree completion at all stages of the model in 
which they were "in play," and 

(b) were most subject to change by external parties with little-to-modest — but creative — 
effort that might improve the portrait of degree completion. 

Five factors stand out, four of which affect small populations in which minority students are 
over-represented. Small populations can add up. These factors are: 

1 . First- year credit generation, i.e., the goal of making sure that postsecondary 
students end their first calendar year of enrollment with 20 or more additive 
credits. 

2. The problem of excessive no-penalty withdrawals and no-credit repeats, which 
affect 10 percent of the cohort. Institutional policy and advising can cut the 
incidence of withdrawals and repeats in half. 

3. Use of summer terms. Strategic enrollment management can move more sections 
of high demand courses into summer terms, offer credit-bearing internships in 
summer terms, and engage in other creative initiatives that will also smooth out 
the utilization of institutional resources over what has become an "academic 
calendar year." 
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4. No delay of entry. This is a matter of recruitment strategy among high school 
students whose commitment to postsecondary education is less than fervid. The 
later they show up, the more their postsecondary fate is in jeopardy. 

5. The high school curriculum component of "Academic Resources." This is 
not a case of "little-to-modest" effort or a small population. It is a megawork in 
progress, much of which depends on students’ reading skills on entering high 
school. If students cannot read close to grade level, the biology textbook, the 
math problems, the history documents, the novel — all will be beyond them. And 
if high schools are not offering a full academic curriculum, there is little hope. 

But with those five factors in mind, and assuming full student response and success, potential 
degree completion rates were hypothesized based on the records of NELS:88/2000 high school 
graduates by race/ethnicity (table 32) and socioeconomic status quintile (table 34). Virtually 
every one of these factors contributed to closing degree completion gaps, but none more than 
high school academic curriculum participation — which, to repeat, is criterion-referenced, hence, 
open to everyone to rank at or near the top. For African-American students, the combination of 
moving into the top 40 percent of the high school academic curriculum intensity index plus 
earning more than four credits during postsecondary summer terms would lower the degree 
completion gap vis-a-vis white and Asian students from 15 percent to 6 percent. For students 
from the lowest socioeconomic status quintile, moving into the top 40 percent of the academic 
curriculum intensity index and entering postsecondary education directly from high school 
would improve degree completion from 36 to 59 percent. For Latino students, the same steps 
would improve degree completion from 45 to 69 percent. Does that mean that future degree 
completion rates will look like those in tables 32 and 34 if everyone meets the criteria on all five 
counts? No; not everybody will make it. But the tables suggest just where the improvements 
could be dramatic — and for whom. 

Messages to Students and Commentators 

Student responsibility (the intersection of choice with opportunity) is a major theme of The 
Toolbox Revisited in a way that was only implicit in the original Tool Box. The essay concludes 
with some recommendations for students, who are partners in their own education fate, who 
shouldn’t wait around for someone else to do something for them, and who are rarely addressed 
in studies of attainment. 

The concluding messages also reflect on the dissonant data of public discourse on high school 
graduation rates, college attrition rates, and college graduation rates, examples of consequent 
"scare stories" that do not help us identify and address real problems, and a plea for creativity 
and cooperation in developing better student tracking systems. These messages also urge a 
considerable change in the language we use in describing what happens to students from a 
negative rhetoric that assumes passivity to one that respects students as active players, seeking 
and discovering paths to their education goals. 
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A Guide for Reading Tables and Terms 



Interchangeable terms 

This essay frequently compares the histories of two grade-cohort longitudinal studies carried out 
by the National Center for Education Statistics (NCES). Each of these studies can be referenced 
in a number of ways, and all of these references are used. 

The following labels for the longitudinal study that began with a national sample of 1 Oth-graders 
in 1980, the High School and Beyond/Sophomore cohort, are interchangeable: 

HS&B/So 

High School Class of 1982 

class of 1982 (where inter-cohort comparisons of 12th-graders are at issue) 

"High School & Beyond/Sophomore cohort" will also be spelled out when the entire longitudinal 
study, including all its data set components, is the subject. 

The following labels for the longitudinal study that began with a national sample of eighth- 
graders in 1988, the National Education Longitudinal Study of 1988, are interchangeable: 

NELS:88/2000 

High School Class of 1992 

class of 1992 (where inter-cohort comparisons of 12th-graders are at issue) 

On reading tables in this study, part 1: Descriptive cross-tabulations 

All tables in this study are constructed to meet the statistical standards for table presentation of 
the National Center for Education Statistics (NCES 2002a). They are stand-alone tables, so that 
if they are reproduced outside the context of the essay, they tell a complete, self-contained story. 
To ensure a complete story, the descriptive cross-tabulations in this document include the 
standard errors of the estimates. The reason for this election — instead of placing tables of 
standard errors in an appendix — is to enable the reader to judge, on the spot, whether the 
difference between any two estimates is statistically significant. While the text draws the 
reader’s attention to statistically significant (and insignificant) estimates important to the 
narrative, it does not comment on all statistically significant estimates. In general, the formula 
invoked for statistical significance is the simple student’s t test: 



t = (Pj - P 2 ) % (sej 2 + se 2 2 ) 



where P, and P 2 are the percentage estimates to be compared and sc, and se 2 are the 
corresponding standard errors. If t >1.96, one has a statistically significant difference at p < .05 
(which means that the probability that this observation would occur by chance is less than 1 in 
20), a standard marker. In the case of multiple comparisons, the critical value for t rises. 2 



2 

“For technical issues concerning standard errors and multiple comparisons, see Appendix D. 
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Variances in estimates 



The reader will often find that estimates for the same phenomenon will differ somewhat, e.g., in 
one table a bachelor’s degree completion rate is 69.3 percent, in another table, 64.6 percent. 

These differences are due to the definition of the universe (i.e., who are we counting?) for each 
table and the weight employed in the calculation. 

On reading tables in this study, part 2: Logistic models 

The core analytic method employed in The Toolbox Revisited is a logistic regression, and there 
are a baker’s dozen tables in both the text and appendices that present the results of logistic 
models. There are many ways in which researchers have presented these results (Peng, So, St. 
John, and Stage 2002), and a number of statistics are employed in these representations. The 
Toolbox Revisited employs the following for each row of these tables: 

The name of the variable — Variable names were created so that the realities to which 
they refer are self-evident. 

The unadjusted parameter estimate — also known as the log odds, or the natural logarithm 
of the odds of the outcome of interest happening, e.g., whether a bachelor’s 
degree was earned. The reader should note whether the sign for this estimate 
indicates a positive or negative relationship to bachelor’s degree completion, as 
well as the magnitude of the estimate (though that alone does not indicate whether 
the estimate is significant). 

The adjusted standard error of measurement for the estimate — Some researchers will say 
that this statistic is not necessary. The author provides it because, without it, one 
could not verify the accuracy of the other statistics on the row. We need the 
adjustment because NCES longitudinal studies are based on complex sampling 
designs. The adjusted standard error is calculated by multiplying the simple 
standard error by the Root Design Effect for students who offer non-missing 
values for all variables in the logistic model at issue (see Appendix D). 

The t statistic — This is the measure that opens the gate to the judgment of degree of 
significance for the variable in question on that row. Again, some researchers 
will say that this statistic is not necessary because in the next column, the p 
statistic indicating degree of significance is provided. The author offers the t 
statistic because a minimum t of 0.765 was selected as a threshold for keeping a 
variable in the sequence of logistic models, and the reader should see those 
instances where a variable fails to meet that criterion. Why 0.765? In any 
statistics textbook, one will find an appendix with a table of relationships between 
levels of significance (p values) and /-values for two-tailed significance tests 
according to the number of degrees of freedom (number of variables in a model 
minus one). If one goes to the column for the lowest level of significance in this 
standard table ( p < .50), and scrolls down to the /-value for the smallest number of 




variables used in any of the logistic models in The Toolbox Revisited, three, one 
will find it equal to 0.765. This is a very generous threshold. 

The p statistic — This notation tells us whether the observed relationship would occur by 
chance, and, if so, at what ratio of chance. A p of <.05 says that the odds of the 
relationship between independent and dependent variables in that model occurr- 
ing by chance are less than 1 in 20. This is also known as a 95 percent confidence 
level. In all of the logistic models in The Toolbox Revisited there are variables 
that do not meet the t requirements for even a marginal level of significance 
( p < . 10). All variables with a p of <. 10 or better are highlighted in bold. 

Delta-p is the critical statistic for telling the story of the association of the parameter 
estimate with degree completion. It says that every unit change in the 
independent variable changes the probability that X will happen by Y 
percent — and the Y is indicated by the Delta-p, e.g., 0.1285 would be translated in 
the text as 12.9 percent. The statement of change in probability is not a linear 
statement, and is always the result of relationships among the variables in a 
particular logistic model. It certainly should not be read as a statement of cause or 
prediction. 

Three decision rules 

In indicating whether a variable was used in the logistic models of The Toolbox Revisited, three 
criteria were observed: 

• Any statistical software package (the Statistical Analysis System, or SAS, was 
used in this report) allows one to set a threshold of statistical significance for a 
variable to be admitted to a multivariate model. The default threshold is that 95 
percent confidence level, represented as p <.05. For both the original Tool Box 
and The Toolbox Revisited this threshold was changed to a much more generous 
.2, i.e., the probability that the variable would be admitted by chance is less than 1 
in 5. The purpose of the more generous threshold is to allow more variables to 
"compete," so to speak, within the boundaries of a statistical model, and then to 
see which variables survive. 

• As noted above, the survival indicator in a model was a t value set to 0.765. If a 
variable failed to exceed that value when utilized in the step of the logistic series 
in which it was introduced, it was not carried forward to subsequent steps. 

• Collinearity — Think of the Venn diagrams you learned in high school 
mathematics. If each circle in the Venn diagram represents a variable, you don’t 
want a situation where two or more of them overlap to a point approaching an 
eclipse, for if they do, analysis of either one is impossible. There are special 
statistics for determining the extent of collinearity, and these were employed to 
determine when this situation arose. When it arose, one of the variables had to be 
dropped. This often happens when one variable is a major component of another, 
e.g., family income in relation to socioeconomic status. The narrative will 
describe these cases. For further comments on collinearity, see Appendix D. 
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Organization of This Data Essay 

The following is a brief outline of The Toolbox Revisited so the reader knows what to expect. 

Part I: Background. The introduction presents the basic question, the data sets invoked, 
the purposes and statistics of the investigation, and the demography of the subject universe. 

Part II: Variables Explored and Used in This Analysis. This short section of the study 
lists all the independent variables that were considered and provides brief definitions and basic 
statistical characteristics. A summary figure (pp. 20-21) indicates which of these met the criteria 
for inclusion in the logistic narrative of Parts III and IV. A more elaborate glossary 
(pp. 179-193) provides details on the construction of these variables, allied data and 
commentary, and will be of particular interest to researchers. 

Part III: What Is and What Happens Before Matriculation. Here we begin the 
chronological narrative, using both descriptive and multivariate data, of what ultimately made a 
difference in bachelor’s degree attainment by December 2000 for 1992 12th-graders who 
attended a four-year college at any time. Part III begins with background demographic 
characteristics, then adds the critical components of high school academic history. 

Part IV: Matriculation and Beyond. This section continues the cumulative steps of the 
logistic narrative, starting with the characteristics of entry to the postsecondary world, and 
continuing with first calendar year perfonnance, financing considerations, attendance patterns, 
and extended performance (that is, taking students’ entire undergraduate careers into account). It 
includes a special consideration for the second calendar year of enrollment and concludes its 
logistic narrative with attention to two very powerful variables: continuous enrollment and the 
ratio of course withdrawals and repeats to the number of all courses attempted. 

Part V: Closing the Gap. Having demonstrated how the universe of independent 
variables is related to degree completion, The Toolbox Revisited then asks two questions: (1) To 
what extent do the major national data sets agree on "graduation rates"? And (2) Given what we 
have learned about what makes a difference in degree completion, what variables provide the 
most promising guidance for closing the gap in graduation rates, by race/ethnicity and 
socioeconomic status, for students who attend a four-year college at any time? 

Part VI: The Missing Element of This Story. A key missing part of the story that is a 
by-product of the limited features of the NELS:88/2000 is addressed in this section: the content 
standards of high school and postsecondary course work. Other brief "excursions" — timing and 
reasons for permanent ("status") drop out from college, and time-to-degree — are placed in 
Appendices H and K respectively. 

Part VII: Messages. Finally, The Toolbox Revisited offers some messages — to students 
and to those who engage in public discourse about the issues we have covered — and highlights 
the major conclusions of the study. 

Appendices: With one exception, appendices are presented in the order in which they are 
cited in the text. The exception is the last appendix, Appendix L, that contains a variety of 
reference tables on miscellaneous topics raised in the text. 
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Part I 

Background 

This study explores the academic resources and momentum students build through their high 
school and college careers, and analyzes the relationships between those factors and degree 
completion rates. It departs from most previous research on attainment by focusing on the 
details of students’ high school and college curricula and academic perfonnance that are 
available from transcript records. Its principal data are drawn from the National Education 
Longitudinal Study of 1988 (hereafter referred to as the NELS:88/2000 or just NELS). This 
longitudinal study followed a national sample of over 12,000 students (representing a weighted 
2.9 million students) from the time they were in the eighth grade in 1988 to roughly age 26 or 27 
in December 2000. 

In round numbers, of the high school graduates in this cohort, 83 percent engaged in some form 
of postsecondary education by age 26, and 68 percent attended a four-year college at some time. 
Of those who earned any credits from four-year colleges at any time (which includes a 
substantial proportion of students who began postsecondary study in community colleges), 66 
percent earned a bachelor’s degree. 

While the 66 percent completion rate sounds impressive for a mass system of higher education, it 
masks an unhappy differential by race/ethnicity, and more so by socioeconomic status. As we 
strive to improve high school graduation rates, to invite greater numbers of high school graduates 
into the postsecondary system, simply to maintain — let alone improve — our completion rates 
will take a great deal of effort. We need constructive guidance and benchmarks. 

This study was designed as a follow-up and replication of a previous attempt to provide that 
guidance — Answers in the Tool Box: Academic Intensity, Attendance Patterns, and Bachelor’s 
Degree Attainment (hereafter referred to as the original Tool Box). 3 Since its publication in 1999, 
the original Tool Box has become one of the most frequently cited works in public discussions 
about — and initiatives to improve — the preparation of students for higher education. Its most 
visible uses have included the presentations of the Texas “Master Scholars” program (2002), the 
revisions in entrance requirements for the University of North Carolina system (2000), standards 
goals set by the Illinois Board of Higher Education (2001), and suggestions of the Education 
Commission of the States for state high school graduation policies (2001). It was summarized in 
major litigation addressed to inequities in opportunity-to-learn in high schools (e.g., Daniel v. 
California 1999), and in the research literature, its analyses of both precollegiate preparation and 
post-matriculation attendance patterns and performance have also been marked (e.g., Horn and 
Kojaku, 2001; Zucker & Dawson, 2001; Cabrera and La Nasa, 2001) and improved on 
(DesJardins, McCall, Ahlburg and Moye, 2002; Cabrera, Burkum, and LaNasa 2005). 



3 

Out of print but available at http://www.ed.gov/students/prep/college/thinkcollege/early/aboutus/edlite- 
resources.html. 
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The analyses in the original Tool Box were based on the High School & Beyond/Sophomore 
cohort longitudinal study (hereafter occasionally referred to as the HS&B/So), the second of the 
U.S. Department of Education’s national grade-cohort longitudinal studies, 4 which followed a 
national sample of lOth-graders from 1980 to 1992 in surveys and to September 1993 on 
postsecondary transcripts. As with all of the grade cohort longitudinal studies carried out by the 
National Center for Education Statistics, 5 test scores are included in the database, along with 
surveys of parents, school teachers, and school administrators. The modal high school graduation 
year for the HS&B/So was 1982, and student age at the conclusion of the study was 29 or 30. 

The HS&B/So data, while compelling, are now somewhat dated. There were considerable 
changes in both the demography and postsecondary entrance behavior of high school seniors in 
the comparatively short span of the decade between 1982 and 1992. Appendix A highlights 
contrasts in selected background characteristics of all 12th-graders in the two longitudinal 
studies cohorts. Some of these changes have been frequently observed (higher proportion of 
minority students, westward and southward movements of populations, higher proportion of high 
school seniors entering higher education and planning to earn a bachelor’s degree), and some 
rarely observed (higher proportion who grew up in mixed-ethnicity neighborhoods, and a higher 
proportion with parents who indicated some postsecondary education). The question naturally 
arises as to whether the hypotheses and analyses based on the history of the High School & 
Beyond/Sophomore cohort would hold up in the story of the NELS:88/2000. The NELS history 
offers a more contemporary account, 6 one that, in terms of secondary school records, may reflect 
the high school curriculum reforms of the mid- and later- 1980s 7 that followed the discussion of 
the report of the National Commission on Excellence in Education, A Nation at Risk (1983). 

The Toolbox Revisited follows the same path of analysis using the NELS:88/2000 as did its 
predecessor with the HS&B/So. In that sense, it is a replication. It is a modified replication, 
however, because it introduces new constructs based on critiques of the original Tool Box and 
because it offers a more refined chronology of steps from high school to the end of a student’s 
undergraduate education. 

A substantial amount of the analyses and endorsements that followed the original Tool Box 
revived the flagging "seamless" K-16 themes of the 1980s post -Nation at Risk refonn effort. In 
an October 2000 policy brief that visited these issues, the Education Commission of the States 
reminded people of what most of the 1980s reform reports did not address: the fact that K-12 and 
postsecondary systems are governed in very different ways, even in the public sector of 
postsecondary, and that there is a consequent "disconnect" of substantial dimensions (Education 



4 The first was the National Longitudinal Study of the High School Class of 1972 (1972-86). 

5 For a description of the grade-cohort studies, see Appendix B. 

6 A fourth-grade-cohort longitudinal study is currently in progress. Called the ELS2002, it began with a 
national sample of lOth-graders in 2002 and is expected to run at least through 2012. We obviously will have to 
wait some years before we see its full results. 

7 For an account of changes in high school graduation requirements in the years immediately following 
issuance of A Nation at Risk, see Stedman and Jordan (1986). 
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Commission of the States 2000). The metaphor of bridges (Venezia, Kirst and Antonio 2003) 
and the rhetoric of disconnect (e.g., Conley 2003) created a more sophisticated focus of analysis 
than the refonn reports of the 1980s embraced. The major work that sought to build new bridges 
and connect the disconnects underscored the obligation of the system not merely to assure 
college "access," but degree completion, with curriculum playing the key role. While "degree 
completion," in public discourse, refers to the bachelor’s degree, the same principles apply to 
associate degrees granted principally by community colleges as well (Adehnan 2005a). 

What Did the Original Tool Box Say? — And Based on What Kind of Evidence? 

In a nutshell, here are the major conclusions of the High School & Beyond/Sophomore-based 
Answers in theTool Box analysis: 

1) Of three traditional measures of precollegiate educational history — curriculum configuration, 
academic performance (on a scale that combines class rank and GPA), and assessed general 
learned abilities (a senior year mini-SAT) — the intensity and quality of one’s secondary school 
curriculum was the strongest influence not merely on college entrance, but more importantly, on 
bachelor’s degree completion for students who attended a four-year college at any time. The 
highest level of mathematics the student reached in high school played a significant role in the 
strength of the curriculum configuration. One of the major contributions of Tool Box not only to 
the literature, but to practice as well, was to change what was understood by the “entry 
characteristics”of students by digging out and demonstrating the power of the academic intensity 
of secondary school curriculum over combinations of standardized test scores and grades. 

2) By moving into the top two quintiles of the curriculum measure and completing a high school 
mathematics course beyond Algebra 2, African-American students who started out in a four-year 
college would hypothetically increase their bachelor’s degree attainment rate from 45 percent to 
73 percent; Latino students who did the same would hypothetically increase their bachelor’s 
degree attainment rate from 61 percent to 79 percent. 8 These increases were significantly greater 
than those for white and Asian students under this scenario, and, more importantly, were 
considerably greater than the influence of moving into the top two quintiles of either test scores 
or class rank/GPA. In other words, curriculum counts, particularly for minority students. 

3) The three traditional precollegiate measures can be combined in a composite measure of 
“Academic Resources.” This measure had continuing, statistically significant relationships with 
bachelor’s degree completion in both linear and (more appropriately and convincingly) logistic 
regression sequences involving four post-matriculation steps (blocks of variables for financial 
aid, attendance patterns, first-year perfonnance, and extended postsecondary performance). 



8 

For African-American students in the HS&B/So who reached the 12th grade and entered postsecondary 
education, 52.3 percent started in four-year colleges, a percentage that rose to 54.2 in the NELS:88/2000. The 
comparable percentages for Latino students, who are more likely to start in community colleges, were 38.9 percent 
for the HS&B/So and 38.6 percent for the NELS:88/2000. For white students, the proportions starting in four-year 
colleges were 54.3 percent for the FIS&B/So and 57.4 percent in the NELS:88/2000. 
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4) Post-matriculation behaviors and attendance patterns that were strongly and positively 
associated with bachelor’s degree attainment were continuous enrollment, transfer from a 
community college to a four-year institution after more than 10 credits earned at the community 
college, and the trend in students’ grades. 

5) Post-matriculation behaviors and attendance patterns that had a strong negative influence on 
bachelor’s degree attainment were the ratio of courses from which the student withdrew or 
repeated to all courses attempted, and earning less than 20 credits in the first calendar year of 
postsecondary attendance. 

6) Socioeconomic status had a modest and diminishing association with bachelor’s degree 
attainment. Minority status had a modest negative association until performance (first-year 
performance and continuing perfonnance) was taken into account, at which point it had no effect 
Gender had no effect at any stage of the model. The only demographic variable to have a strong 
(and in this case, negative) association with degree completion was becoming a parent by age 20. 

The overall message was about academic momentum and what adds to that momentum at each 
stage of a student’s history from secondary school onward. The evidence was archival, was 
treated in the tradition of quantitative history (Elder, Pavalko, and Clipp 1993; Clubb, Austin and 
Kirk 1989; Haskins and Jeffrey 1990), and like this essay, did not claim causality. 

Why use these data sets? 

There are three types of national data sets available to construct longitudinal analyses such as the 
original Tool Box and this replication, but only one type of data set — the NCES transcript-based 
grade-cohort study — is truly suited to the task. The other two are (1) the Cooperative 
Institutional Research Project (CIRP) occasional longitudinal follow-ups to its annual survey of 
entering college freshmen, and (2) the NCES Beginning Postsecondary Students studies (BPS). 
Each of these has its virtues, and will be revisited in Part V of this study when we compare what 
three different longitudinal studies of the 1990s say about degree-completion rates. The CIRP 
produces an enonnous amount of information on student attitudes, values, and college 
experiences, and does so with large samples of (principally) entering four-year college students. 
Its occasional longitudinal follow-ups involve sufficient history (e.g., six or nine years) to track 
not only long-tenn undergraduate completion rates, but also postbaccalaureate education (Astin 
1993). The BPS longitudinal studies are shorter (five or six years), not dependent on institutional 
decisions to participate (as is the CIRP), inclusive of students of all ages at entry, and, as befits 
their principal population sample (a subset of the triennial National Postsecondary Student Aid 
Study), contain very strong and reliable financial aid data. The BPS study of 1995/96-2001 will 
be used at a number of points in this study to expand the range of our observations. 

However, in both cases, all features of precollegiate history must be rendered as exogenous 
variables. The high school histories provided by students in CIRP are retrospective, and the 
precollegiate histories in the Beginning Postsecondary Students studies derive from a 
combination of retrospective offerings by students and accounts on the student information 
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questionnaires for those who took either the Scholastic Assessment Test (SAT) or American 
College Test (ACT) within the two years prior to the BPS start date (nearly half the students in 
the BPS studies did not take either exam). As Kahn and Nauta (2001) demonstrated, there is an 
inevitable loss of accuracy in the process of these retrospections. 

On the other hand, the very nature of a longitudinal study population assembled in the 10th grade 
(High School & Beyond/Sophomore cohort) or eighth grade (NELS:88/2000) renders 
precollegiate history endogenous to analytic models. The transcript-base overrides student 
accounts and provides far more detail than either paper-and-pencil surveys (used by CIRP) or 
computer-assisted telephone interviews (used by the BPS studies) can provide. 

Every data set sacrifices something. No data set is constructed with the questions a particular 
researcher may have a decade later, so variables are derived and secondary. The High School & 
Beyond/Sophomore study and the NELS:88/2000 are wanting in many ways where the other 
studies are strong: financial aid (BPS), and changes in values and opinions (CIRP). But as 
stories about the core activities we call education, they are unsurpassed by the others. 

Purposes and Statistics of This Monograph 

The primary purpose of this monograph is to trace the elements of academic momentum as they 
played out in the secondary school and college history of the High School Class of 1992 
(through December 2000) compared with the parallel history of the High School Class of 1982 
that was the foundation for the original Tool Box study. In the process, the analysis is enriched 
by including variables we learned to construct or modify on the basis of commentaries and 
critiques of Tool Box, and by a more accurate chronological order in presentation of those 
variables. The portrait of academic momentum that emerges is a framework within which more 
sophisticated analyses can be pursued, and within which ameliorative policies (the tools) can be 
advocated. This study does not pretend to answer complex questions about indirect effects of 
home, peer, school, and postsecondary institution interactions, but rather trusts future research to 
deal with those issues. 

A secondary task is to demonstrate the construction of a replication when the two data sources, a 
decade apart, are presumably parallel, but turn out to be something less (for a key example, see 
Appendix C). The principal encouragement for the replication is that the bachelor’s degree 
attainment rate for students who attended a four-year college at any time remained the same 
despite differences in the length of cohort history: 65.6 percent for the HS&B/So over 1 1 years, 
and 66.5 percent for the NELS:88/2000 cohort over 8.5 years (Adehnan 2004a, table 2.2, p. 21). 
If the HS&Beyond/So history were truncated at 8.5 years, the bachelor’s degree attainment rate 
would have been 59.7 percent. From this perspective there was a marked improvement in degree 
completion for traditional age college students over the two decades in question. It is natural to 
ask how this happened, and if the answers to that question provide any guidance for the future. 
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In terms of statistical technique, both the original Tool Box and The Toolbox Revisited use simple 
logistic regression, not structural equations or other path models that are common to causal 
inquiries or searches for indirect effects, e.g., of discrete aspects of school or college 
environments (Dey and Astin 1993). A logistic regression is focused on an event that either 
happens or it doesn’t. The dependent variable is dichotomous: yes or no. The independent 
variables are judged within each model by the degree to which they contribute to what happened 
in relation to or controlling for all other independent variables in the model (Hair, Anderson, 
Tatham, and Black 1995). 

There are a number of ways of expressing this “degree.” One is by an “odds ratio,” which, 
expressed in a simple way, is a ratio of the odds that X will happen given a unit of change in the 
independent variable to the odds of X not happening, and ultimately shows the strength of 
association between the independent and dependent variables — with the closer the odds ratio 
to 1, the less the strength of the association. 9 This was the measure used in the original Tool 
Box. Another way of expressing the value of the contribution of an independent variable is by a 
“Delta-p” statistic that says every unit change in the independent variable changes the 
probability that X will happen by Y percent given the values of the other variables in the model 
(Peterson 1985; Cabrera 1994). The narrative of The Toolbox Revisited relies on Delta-p, 10 and 
the logistic model tables provide Delta-p statistics only for those parameter estimates that are 
statistically significant since there is no way to determine the statistical significance of the Delta- 
p itself (Cabrera 1994). 11 

But in this paper there is a major methodological departure from the original Tool Box study: 
there are seven (and not five) steps in the model employed, all driven by the empirical history of 
the NELS:88/2000 students. Following St. John, Paulsen, and Starkey (1996), the blocks of 
variables in each step were entered "in a sequence that parallels the order in which students pass 
through well-established stages of persistence behavior" (p. 194) on their way toward bachelor’s 
degree completion (or not). Each of the seven steps, too, is cumulative. That is, variables in one 
step that meet the statistical criteria for remaining in the model are carried forward to the next 
step. This extended accounting, which we will call a "logistic narrative," allows "a meaningful 
examination of the direct effects of variables on persistence, as well as their interactions with the 
variables entered in successive steps" (St. John, Paulsen, and Starkey 1996, p. 194). 

The reader can already tell that there is a great deal of technical material in this presentation, but 
it is presented in the spirit of the U.S. Department of Education’s goal of building a culture of 
evidence. The author trusts that reports such as The Toolbox Revisited will contribute to the 



9 Odds ratios, as Peng, So, Stage, and St. John (2002) remind us, are not odds. Their interpretation is not as 
transparent as the original Tool Box assumed them to be. And while an R 2 statistic is presented in the logistic tables 
of this study, it cannot be read the way one would interpret an R 2 in a linear regression, i.e., it does not indicate the 
percent of variance in the dependent variable that is explained by the independent variables (Long, 1997). For that 
reason, it is called a "pseudo R 2 " (Cabrera 1994) and is one of a number of measures of goodness-of-fit. As blocks 
of variables are added to the model in the stepwise manner followed here, the pseudo R 2 should increase. 

10 For the computation of Delta-p, I am using a shortcut recommended by Paul Allison of the University of 
Pennsylvania: Z?p( 1 -p), where b is the logistic coefficient and p is the probability for the dependent variable in the 
model. This heuristic produces slightly higher values than the formula advanced by Petersen (1985). 

1 ] For all technical issues, please see Appendix D. 
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attainment of that goal. Some of the technical material is placed in appendices so that 
researchers have access to documentation, and all readers have access to reference material. 

Who Are We Talking About? And Who Are We NOT Talking About? 

As was the case for the original Tool Box, the basic universe of analysis in The Toolbox Revisited 
does not consist of everybody who started out in the cohort. We have to be very clear about this. 
The basic question of both studies is: 

What demographic, high school performance, postsecondary entry, and 
postsecondary history (attendance patterns, academic performance) factors are 
convincingly associated with bachelor’s degree attainment for 12th-graders who 
subsequently attended a four-year college at any time in their undergraduate careers? 

This is not a question about completing high school, completing high school on time, or 
completing high school on time with a standard diploma (as opposed to a GED or certificate of 
attendance). 12 

This is not a question about entering the postsecondary system. We are not talking about 
"access." Nor is it a question about "persistence" to the second tenn or the second year 
following entry to postsecondary education. It certainly is not about "retention" in the same 
institution to the second term or the second year. 

This is a question about completion of academic credentials — the culmination of opportunity, 
advisement, choice, effort, and commitment. 

The question is carefully worded. The universe does not include people who never reached the 
12th grade. In fact, as a consequence of the sampling design and weighting of the two 
longitudinal studies, both essays include only those students who were in the 12th grade in the 
same year, 13 along with those who graduated from high school. The definition of the universe 
also requires that we have demographic information, high school transcripts, 14 test scores, and 
postsecondary transcripts for everybody whose careers are subject to analysis. Not everybody 
can present all this information, and, with rare exceptions (involving some aspects of high school 
records, and as explained in Appendix C) both the original Tool Box and The Toolbox Revisited 
decline to impute any of this information. 



12 

“For details on the high school graduation status of those who were eighth-graders in 1988, see Appendix 
L, table LI. 

I3 For details on those in the NELS:88/2000 cohort who, though in the 12th grade in 1992 along with 
others, had previously been held back at least one year in the course of their schooling, see Appendix L, table L2. 

14 Only 21.3 percent (s.e. = 1.17) of the NELS:88/2000 who were in the 12th grade in 1992 were missing 
high school transcripts, compared with 73.7 percent (s.e. = 2.58) of those who had either dropped out of high school 
or were not in-grade. Nearly 100 percent of the former group graduated from high school with a standard diploma. 



9 




There are two other notable features of the basic question. First, both essays argue that, 
regardless of what students say on a survey about their education expectations, it is what they 
actually do that counts. So the only students we can talk about honestly with respect to 
bachelor’s degree attainment are those who set foot in a bachelor’s degree-granting institution at 
some time. This criterion obviously includes students who started their postsecondary careers in 
community colleges and other types of sub-baccalaureate institutions. Second, the term used to 
describe the relationship between the dozens of independent variables describing demography, 
high school performance, postsecondary entry, and postsecondary history and the dependent 
variable of bachelor’s degree completion is "convincingly associated." Neither essay claims 
"cause" or "prediction." 

To illustrate how these conditions restrict the universe, let us start with 2.93 million 1988 eighth- 
graders in the NELS:88/2000 cohort. Table 1 lays out the stages of contraction of the basic 
population until we reach the subject universe of this study. 

Table 1. From macro to micro: Contraction of the universe of 1988 eighth- graders to 
the universe subject to analysis in The Toolbox Revisited 



Description of Universe 


Percent 


Descending 
weighted N a 


A. Initial universe of 1988 eighth-graders 


100.0 


2.93M 


B. Of (A), those who were in the 12th grade in 1992 


83.6(0.98) 


2.45M 


C. Of (B), those who continued to postsecondary 
education at any time through December 2000 


81.7(1.28) 


2.0M 


D. Of (C), those who presented complete high 
school transcripts, test scores, b complete postsecondary 
transcript records, and socioeconomic status infonnation 


80.5 (1.01) 


1.61M 


E. Of (D), those who attended a four-year college 
at any time. 


73.5 (1.00) 


1.19M 


Net percentage of 1988 eighth-graders in the universe 
Net percentage of 1992 12th-graders in the universe 


41 

51 


1.19M 

1.19M 



“Like other NCES longitudinal studies, the NELS:88/2000 cohort is a stratified sample, in which each student is 
assigned a weight to represent other similar students in the cohort (see Curtin, Ingels, Wu, and Heuer 2002). 
b See definition of SRTSQUIN in Glossary. 

NOTES: Standard errors are in parentheses. 

SOURCE: National Center for Education Statistics: NELS:88/2000 postsecondary transcript files (NCES 2003-402 
and Supplement). 
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Looking more closely at our universe in terms of the types of schools attended, we find that 
20.2 percent (s.e. = 1.13) started in community colleges, and 41.0 percent (s.e. = 1.29) earned 
credits from community colleges, whether or not they started at community colleges. While we 
will describe the attendance patterns of the NELS:88/2000 cohort in more detail later, the point 
is that the universe under analysis is not limited to those who attended only four-year 
institutions, and that the community college plays a significant role in the careers of the group of 
students under consideration. 

To repeat: Our subject universe consists of half the 1992 12th-grade students in the 
NELS:88/2000 cohort. We are not talking about: 

• students who did not graduate from high school or those who graduated with 
something other than a standard diploma (G.E.D. or Certificate of Attendance); 

• students for whom we do not have full high school records (transcripts, grades, 
test scores); 

• students for whom socioeconomic status could not be determined; 

• students who did not enter any postsecondary institution by December 2000, 
when they were 26 or 27 years old; and 

• students who entered the postsecondary system, but never set foot in a bachelor’s 
degree-granting institution. 

That is the other half. Some 22.2 per cent (s.e. = 1.05) of 1992 12th-graders who attended a 
four-year college at any time are in this “other half’ because they are missing key infonnation 
that excludes them from the universe under analysis. Readers interested in the demographic 
differences between included and excluded students due to missing data elements are referred to 
Appendix E. 

Comparative demography of the subject universe 

Given the strong boundaries drawn around the subject universe, how does its demography 
compare to less restrictive definitions? Table 2 sets out the major demographic categories, and 
compares the target subject universe with (a) all 1988 eighth-graders who participated in the last 
survey of the NELS:88/2000 in 2000, (b) all participants in the 1992 survey, whether or not they 
were in school or in the 12th grade, 15 and (c) all 1992 12th-graders who attended any 
postsecondary institution by December 2000. 

As readers work from left to right across the columns of table 2, they will notice that gender 
balances are even until the moment of entry to postsecondary education, at which point women 
pull ahead of men. The same type of observation applies to the race/ethnicity distribution: At the 
point of entry to postsecondary education, the proportion of African-Americans, Latinos, and 
American Indians declines, while that of whites and Asians rises. In the universe for this study, 



15 The reader should note that, unlike the High School & Beyond/Sophomore cohort study, the 
NELS:88/2000 sample was "refreshed" in 1992 to be representative of 12th-graders in that year. The "refreshed" 
students are included in our analyses. 
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Table 2. For each of four definitions of the universe of students in the NELS:88/2000, 

percentage distribution by gender, race/ethnicity, socioeconomic status quintile, 
and second language background 

1992 12th-graders 









All 1992 


who attended a 








12th-graders 


four-year college at 




1988 


All 1992 


who entered 


any time and met 


Demographic 


eighth- 


survey 


postsecondary 


other criteria to be 


variable 


graders 


participants 


education 


subjects of this study 1 


Gender 










Men 


49.7(1.01) 


49.9(0.83) 


46.5 (0.93) 


48.8 (1.27) 


Women 


50.3 (1.01) 


50.1 (0.83) 


53.5 (0.93) 


51.2(1.27) 


Race/ethnicity 










White 


71.7(1.50) 


71.5 (1.30) 


74.9(1.29) 


78.2 (1.31) 


African-American 


12.9(1.26) 


12.7 (0.94) 


10.3 (0.90) 


9.4(1.03) 


Latino 


10.5 (0.87) 


10.4 (0.84) 


9.1 (0.88) 


7.0 (0.72) 


Asian 


3.5 (0.32) 


3.7(0.31) 


4.8 (0.43) 


4.7 (0.42) 


American Indian 


1.4 (0.43) 


1.7(0.55) 


0.7 (0.23) 


0.6(0.18) 


Second language 
background 










Nonnative speaker 
of English 
Native speaker of 


8.6 (0.68) 


10.1 (0.83) 


10.0 (0.90) 


7.4 (0.67) 


English from a 
second language 


3.3 (0.33) 


2.7(0.21) 


2.4 (0.23) 


2.2 (0.28) 



household 



Socioeconomic 



status quintile 



Highest quintile 21.3 (0.92) 21.1 (0.88) 29.1 (1.08) 38.5 (1.52) 

2nd quintile 20.8 (0.79) 21.0(0.69) 25.3 (0.88) 26.4 (1.24) 

3rd quintile 20.7(1.10) 19.8 (0.68) 20.2(0.73) 17.7 (0.85) 

4th quintile 19.6(0.83) 19.2(0.66) 15.4(0.61) 11.7 (0.59) 

Lowest quintile 17.6(0.93) 18.9(0.85) 10.0 (0.73) 6.8 (0.50) 

a 1992 12th-graders with known socioeconomic status and high school records (transcripts and test scores), who 
graduated from high school by December 1996, and attended a four-year college at any time. 

NOTES: Standard errors are in parentheses. Columns for gender, race/ethnicity, and socioeconomic status quintile 
may not add to 100.0 percent due to rounding. 

SOURCE: National Center for Education Statistics: NELS:88/2000 postsecondary transcript fdes (NCES 2003-402 
and Supplement). 
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the proportion of whites rises even further, while that of Asians, American Indians, and African- 
Americans 16 holds steady, and that of Latinos declines. There are no statistically significant 
differences in the proportion of students who were nonnative speakers of English 17 until the final 
boundaries are drawn for the universe for this study. 

The most dramatic and expected changes in the distributions of table 2 are by socioeconomic 
status quintile, with quantum leaps in the proportion of students from the top two quintiles as 
soon as one crosses the postsecondary line, and parallel declines in the proportion of students 
from the bottom two quintiles. Only the third quintile (the 41st-60th percentile) remains stable 
until the final contraction of the cohort for the universe used in this study. 

What do these demographic changes within the universes of students who might be the subjects 
for a study mean? The Toolbox Revisited — and Answers in the Tool Box before it — is a study 
about students who graduate from high school and go on to college. Given historical data, it is 
not surprising that this group will evidence a higher SES profile and a lower percentage of 
under-represented minorities than others in the same grade cohort. Since the 1980s, women have 
been in the majority in this group. 18 Within our subject universe, demography is not necessarily 
destiny, as the original Tool Box demonstrated. For the comparison group — the other half that 
either does not graduate from high school, graduates but doesn’t continue its education, or 
graduates and continues but never at a four-year college — The Toolbox Revisited says: If we 
want more people to wind up with at least a bachelor’s degree in life, here are some guidelines 
based on the experience of people who tried. 

Summary of Part I 

What do Answers in the Tool Box and The Toolbox Revisited do, and what do they not do? Both 
studies use the histories of 12th-graders who subsequently attended a four-year college at any 
time (entered in any term and not just the fall term, entered as part-time as well as full-time 
students, entered in community colleges as well as four-year institutions) to indicate what factors 
in those histories are associated with completing a bachelor’s degree — not in four years, not in 
six years, but whenever the longitudinal tracking ended (8.5 years from high school graduation 
for the class of 1992; 1 1 years for the class of 1982). They do not ascribe cause or pretend to 
predict. They recognize that what is associated with degree completion in one generation may 
not be associated with it in the next, or that the strength of association may change. Conditions 
and populations change, after all. Rigid prediction is a risky call, and besides, that’s not what the 
data and statistical standards allow one to do. 



16 The difference in the African-American proportion of the two universes is not statistically significant. 

17 

Nonnative speakers were defined as those who said their first language was not English and, in grade 10 
and/or 12 (the 1992 survey) indicated that they spoke to their mothers most or all of the time in their first language. 

18 

Though to the extent to which the group includes students who commenced their postsecondary studies 
in community colleges — and it does — that majority will be a little less. 
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Part II 

Variables Explored and Used in This Analysis 



With two major exceptions — high school academic curriculum intensity 
and the composite high school performance variable we call "Academic 
Resources," both of which are covered in Part III — the construction and 
allied information for most of the variables explored in The Toolbox 
Revisited are detailed in the Glossary (starting on p. 179). Whenever the 
reader asks for details on "How was that defined?" or “How did they get 
that?” the Glossary is the place to go. 



The purpose of this section is to provide the reader with an advance reference list of the variables 
considered and used in this analysis. The reader should keep in mind that “variables” are 
representations of realities (e.g., first-year college grades) or constructs (e.g., transfer). We use 
them as a shorthand. 

The variables listed and described are presented in the order of the steps of the logistic narrative 
for which they were considered. The collection represents neither a "fishing expedition" nor a 
"mind dump," as a majority of the variables considered were determined by the attempt to 
replicate the original Tool Box study with a more recent parallel cohort. A quick summary of 
their properties can be found in figure 1 . 

Demographic background variables defined 

1) NNSE — Nonnative speaker of English. A dichotomous variable marking those students 
whose first language was not English and who, in grade 12, reported that they conversed in that 
language with their mothers most or all of the time. 

2) IMMIG — Parent immigrant status. A dichotomous variable indicating whether the student’s 
parents were immigrants to the United States within the previous 10 years. 

3) BROSIS — Number of siblings. A dichotomous variable marking students with three or more 
siblings, versus those with one, two, or none. 

4) FIRSTGEN — First generation postsecondary student. A dichotomous variable indicating 
students for whom neither father nor mother ever attended a postsecondary institution. 

5) FAMING — Family income. This variable was first set in six bands, then trichotomized to 
yield upper-, mid-range, and low-income populations. 
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6) URBAN — A dichotomous variable indicating whether the student’s high school was located 
in an urban area. 

7) NEWCHILD — A dichotomous variable to mark students, male or female, who became 
parents by the time they were 20 years old. 

8) RACE — A dichotomous race/ethnicity variable, with under-represented minority (African- 
American, Latino, and American Indian) = 1 and White and Asian = 0. 

9) GENDER — A dichotomous variable in which male = 1 and female = 0. 

10) SESQUINT — Socioeconomic status quintile. See Glossary. 

High school background variables defined 

1) EDU ANTIC — Education anticipations. A three-level variable: (a) consistently (in grades 10 
and 12) expected to earn a bachelor’s degree; (b) raised expectations to the bachelor’s degree 
between grades 10 and 12; and (c) either lowered expectations from the bachelor’s level between 
grades 10 and 12 or never expected to earn a bachelor’s degree. 

2) CLSSRNKQ — High school class rank/GPA quintile. See Glossary and Appendix C for 
accounts of construction and limited imputation, respectively. 

3) SRTSQUIN — Senior year test score quintile. A test of general learned abilities was 
administered to NELS high school seniors and the results set out in percentiles. For those who 
did not take that test but for whom SAT or ACT scores were available, those scores were 
substituted using an equipercentile concordance methodology, weighted, and set out in quintiles. 
See Glossary for a more detailed account. 

4) HIGHMATH — Highest level of mathematics reached in high school. A five-level variable 
with calculus and precalculus at the high end and Algebra 1 and prealgebra at the low end. See 
Glossary. 

5) SCIMOM — High school momentum in science and mathematics. A three-level variable 
combining highest level of mathematics with numbers of credits earned in core laboratory 
science. See Glossary. 

6) FLAN — Number of units of foreign language in high school on a five-level scale. 

7) ADVANCE — Number of Advanced Placement (AP) courses. Three values based simply on 
the number of AP courses recorded: three or more, one or two, and none. See Glossary for a full 
account of identifying AP course work. 
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8) HSCURRQ — Academic intensity of high school curriculum, in quintiles. This variable is the 
core of the analysis of students’ precollegiate histories, and, hence, is described in detail in 
Part III and Appendix F. 

9) ACRES — Academic Resources. A quintile index representing a composite of students’ pre- 
collegiate attainment (curriculum plus class rank/GPA plus senior year test score). ACRES is the 
dominant precollegiate variable in both the original Tool Box and The Toolbox Revisited. The 
construction of this variable is described in detail in Part III. 

Postsecondary entry variables defined 

1) FIRST4 — A dichotomous variable indicating whether the first postsecondary institution 
attended by the student was a four-year college. 

2) DOCT — Another dichotomous variable indicating whether the first postsecondary institution 
attended was a doctoral degree-granting institution. 

3) SELECT — A dichotomous variable indicating that the first institution attended by the student 
was either highly selective or selective. 19 See Glossary for details on selectivity. 

4) NODELAY — A dichotomous variable marking students who entered postsecondary education 
within seven months of high school graduation. 

5) ACCELCRD — Acceleration credits. A sum of all college credits earned by both course work 
prior to high school graduation and by examination. The values of this variable were set at three 
levels: more than 4 credits, 1-4, and zero. 

First- year performance variables defined 

1) LOWCRED — A dichotomous variable marking students who earned less than 20 additive 
credits earned in the first calendar year of attendance. For descriptive data on the relationship 
between number of credits earned in the first year and highest degree, see Appendix L, table L4. 

2) FRSHGRAD — GPA in the first calendar year of attendance. Grade point averages were 
determined for the first full calendar year of postsecondary attendance and were then set out in 
quintiles. FRSHGRAD is a dichotomous variable that divides the highest two quintiles from the 
other three. 



^Institutional selectivity in the postsecondary files of all three grade-cohort longitudinal studies conducted 
by the National Center for Education Statistics has five values: highly selective, selective, nonselective, open-door, 
and not ratable. The first three of these values were based on the selectivity cells developed by the Cooperative 
Institutional Research Project (CIRP) at UCLA for its annual survey (since 1966) or entering freshmen. For the 
distribution of NELS:88/2000 postsecondary students by selectivity of first institution attended, see Appendix L, 
table L3. 
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3) FREM — A dichotomous variable indicating any remedial course work in the first calendar 
year of attendance. 

4) FCOLMATH — Another dichotomous variable indicating whether the student earned any 
credits in college-level mathematics during the calendar year following first enrollment. 
"College-level mathematics" was defined to include college algebra, finite math, statistics, 
precalculus, calculus, 20 and a category of "liberal arts mathematics" courses that require the 
equivalent of high school Algebra 2 as a prerequisite, and include such topics as game theory, 
basic combinatorics, and foundations of numerical methods. 

Financing postsecondary schooling variables 

There are three dichotomous financing variables in both Answers in the Tool Box and The 
Toolbox Revisited : GRANTS, LOANS, and STUWORK. Each indicates only whether the 
student used that form of financing during his/her early postsecondary career. The reader is 
referred to the text of Part IV for elaboration. 

Attendance pattern variables defined 

1) TRANSFER — community college to four-year. With students going back and forth between 
community colleges and four-year colleges, it is important to mark transfer as a pennanent 
change of venue, a migration that is formally recognized by system rules. A transfer student is 
one who (a) started in a community college, (b) earned more than 10 credits from the community 
college before (c) enrolling in a four-year college and (d) earning more than 10 credits from the 
four-year college. The only time limit set for these changes of venue and credit accumulation is 
the length of the longitudinal study. In the case of the NELS:88/2000, that means 8.5 years from 
the modal high school graduation data of June 1992. This is a dichotomous variable. 

2) FOURTRAN — A dichotomous variable indicating transfer from one four-year college to 
another. The algorithm for classic transfer from a community college to a four-year college was 
fairly easy to construct. But to distinguish a true four-year-to-four-year college transfer required 
an indirect route. See Glossary for details. 

3) MULTINS — A dichotomous variable indicating that the student attended more than one 
institution. This is a macro-vision of otherwise multidirectional student behavior. 

4) SUMMER — Number of credits earned during summer terms, set in three bands: more than 4, 
1-4, and none. 



20 

The category of "calculus" includes not only the standard sequence but also special "short calculus" or 
"brief calculus" courses, and specialized introductions to calculus for business, economics, and life science majors. 
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5) PARTTIME — A dichotomous variable tagging students whose enrollment intensity was ever 
part-time. The construction of this variable involves complex algorithms, and the reader is 
referred to the Glossary for details. (For an account of enrollment intensity in both the 
NELS:88/2000 and the Beginning Postsecondary Students longitudinal study of 1995/96-2001, 
see Appendix L, table L5). 

Extended postsecondary performance variables defined 

1) TREND — Trend in student’s GPA: rising, flat, or falling. Cumulative undergraduate GPA 
was measured at three points in time — at the end of the first calendar year following initial 
enrollment, at the end of the first two calendar years following initial enrollment, and at the end 
of the student’s undergraduate career, no matter when that occurred. See Glossary for further 
elaboration. 

2) CUMMATH — Number of credits earned in college-level mathematics: more than 4, 1-4, 
and 0. 

3) CHANMAJ — A dichotomous change-of-major variable. See Glossary for a full description. 

Final factors variables defined 

1) NOSTOP — In all three NCES postsecondary transcript-based grade-cohort studies, 
noncontinuous enrollment was defined as more than a one semester (or its equivalent, e.g., two 
quarters) stop-out period. In the dichotomous variable, NOSTOP, the student is considered 
continuously enrolled even with one semester (or two quarters) off. 

2) WRPT Ratio — A dichotomous variable. On one side of the dividing line are students who 
withdrew from or repeated 20 percent or more of all courses in which they enrolled (ratio of non- 
penalty withdrawal and no-credit repeat grades to all grades received). 

Summary: Locus of Responsibility 

Figure 1 presents the basic statistical characteristics for 39 of the major variables described 
above that were either tested, used in trials, and/or employed in the final logistic narrative of The 
Toolbox Revisited. Seven of these variables describe demographic characteristics, and another 
three label types of financial aid that ultimately are offshoots of demography. Of the remaining 
29 variables, 14 are principally matters of student choice (e.g., changed major), five are 
indicators of student academic effort (e.g., first-year grades), seven reflect interactions of student 
choice plus student effort plus opportunity to leam (e.g., highest high school math), two mark 
interactions of student effort and institutional judgment or guidance (e.g., first-year remediation), 
and one (education anticipations) reflects student experience, attainment, and self-assessment, 
along with the encouragement of family and peers. The student is at the center of all these 
representations. The locus of responsibility for the way each of these variables will tilt lies as 
much with the student as with external forces. The Toolbox Revisited is optimistic that most 
students can make it all come out right. We will return to this optimism in Part VII. 



19 




Figure 1. Values, means, standard deviations, use of variables considered, and step no., if used, in the logistic narrative of 
The Toolbox Revisited , with universe confined to 1992 12th-graders with complete high school records and known 
socioeconomic status who attended a four-year college at any time through December 2000 



Variable 


Values 


Programming 

name 


Mean 


S.D. a 


Used 


Step no. 


Relationship to variables 
in the original Tool Box 


Nonnative speaker of English 


2 h 


NNSE 


0.1472 


0.3544 


No 




New 


First generation student 


2 


FIRSTGEN 


0.1664 


0.3725 


Trial only 




New 


Number of siblings (more than 2 = 1) 


2 


BROSIS 


0.2912 


0.4543 


Trial only 




New 


Race/ethnicity (minority =1) 


2 


RACE 


0.1678 


0.3737 


Yes 


1-5 


Carried forward 


Gender (male = 1) 


2 


GENDER 


0.4677 


0.4990 


Yes 


1-7 


Carried forward 


Socioeconomic status quintile 


5 


SESQUINT 


3.6909 


1.2840 


Yes 


1-7 


Carried forward 


Family income 


3 


FAMINC 


4.0792 


1.5641 


Trial Only 




New 


Became parent by age 20 


2 


PARENT 


0.0282 


0.1655 


Yes 


1-7 


Carried forward 


Education anticipations 


3 


EDUANTIC 


2.8747 


0.4120 


Yes 


1-7 


Modified 


(consistency and level) 


Highest high school math 


5 


HIGHMATH 


3.0030 


1.3560 


Trial only 




Carried forward 


(prealgebra to calculus) 


High school science momentum 


3 


SCIMOM 


2.1146 


0.8837 


Trial only 




New 


High school foreign language units 


5 


FLAN 


3.3662 


1.1617 


Trial only 




New 


Advanced Placement 


3 


ADVANCED 


1.2493 


0.5833 


Trial only 




New 


High school curriculum intensity 


5 


HSCURRQ 


3.7424 


1.1860 


Trial only 




New 


Senior year test score quintile 


5 


SRTESTQ 


3.8533 


1.1752 


Trial only 




New 


High school class rank/GPA quintile 


5 


CLSSRNKQ 


3.6272 


1.2965 


Trial only 




New 


Academic Resources quintile 


5 


ACRES 


3.6703 


1.2130 


Yes 


1-7 


Carried forward 


No delay of postsecondary entry 


2 


NODELAY 


0.9351 


0.2463 


Yes 


2-7 


Carried forward 


Acceleration credits 


3 


ACCEL 


1.4076 


0.7546 


Yes 


2 


New 



See notes at end of figure. 
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Figure 1. Values, means, standard deviations, and use of variables considered, and step no., if used, in the logistic narrative of 
The Toolbox Revisited, with universe confined to 1992 12th-graders with complete high school records and known 
socioeconomic status who attended a four-year college at any time through December 2000 — continued 



Programming Relationship to variables 



Variable 


Values 


name 


Mean 


S.D. a 


Used 


Step # 


in Answers in the Tool Box 


First school was a four-year 


2 


FIRST4 


0.7842 


0.4114 


No 




Carried forward 


First school was doctoral 


2 


DOCT 


0.3764 


0.4845 


No 




Carried forward 


First school was selective 


2 


SELECT 


0.2348 


0.4239 


Yes 


1-7 


Carried forward 


Less than 20 credits in first calendar 


year 2 


LOWCRED 


0.2045 


0.4034 


Yes 


3-7 


Carried forward 


First calendar year grades 


2 


FRESHGRD 


0.4320 


0.4954 


Yes 


3-7 


Carried forward 


Any first calendar year remediation 


2 


FREM 


0.2204 


0.4146 


Yes 


3-6 


New 


Any first year college-level math 


2 


FCOLMTH 


0.5294 


0.4992 


Yes 


3-5 


New 


Grants/ scholarship s 


2 


GRANTS 


0.4972 


0.5000 


No 




Carried forward 


Loans 


2 


LOANS 


0.3180 


0.4658 


No 




Carried forward 


Work-Study/campus job 


2 


ST U WORK 


0.3016 


0.4590 


Yes 


4 


Carried forward 


Attended multiple institutions 


2 


MULTINS 


0.6180 


0.4859 


Yes 


5-7 


Modified 


Community college transfer 


2 


TRANSFER 


0.1377 


0.3446 


Yes 


5-7 


Carried forward 


Four-year to four-year transfer 


2 


FOURTRAN 


0.1476 


0.3547 


Yes 


5-7 


New 


Summer-term credits 


3 


SUMMER 


2.1577 


0.9184 


Yes 


5-7 


New 


Ever part-time 


2 


EVERPT 


0.3342 


0.4717 


Yes 


5-7 


New 


Trend in GPA (rising, flat, falling) 


3 


TREND 


2.1733 


0.7311 


Yes 


6-7 


Modified 


All credits in college-level math 


3 


MATH 


2.2305 


0.8388 


Yes 


6-7 


New 


Changed major 


2 


CHANMAJ 


0.4765 


0.4995 


No 




New 


Continuous enrollment 


2 


CONTIN 


0.7984 


0.4012 


Yes 


7 


Carried forward 


Ratio of withdrawal/repeat grades 
to all courses attempted 


2 


WRPRATIO 


0.0905 


0.2870 


Yes 


7 


Carried forward 


Remedial problem (type and 
amount of remediation) 


5 


REMPROB 


3.8960 


1.4730 


No 




Carried forward 



a Standard deviation. 

b A value of 2 indicates a dichotomous variable, i.e. yes (1) and no (0). 

SOURCE: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402 and Supplement) 
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Part III 

What Is and What Happens Before Matriculation 



There are at least three distinct ways of setting up Step 1 in the progression of multivariate 
analyses that lead us to appreciate what makes a difference (and how much of a difference) in 
completing a bachelor’s degree for students who attended a four-year college at any time. The 
first step covers both student demographics and high school performance. Let us cover the 
demographics first because they do not present complex analytic choices. 

What If We Knew Nothing Except Demography? 

The demographic background of students is marked as of a set moment in time (in our case, in 
grade 12). It is what students look like, where they are living, their parents’ education(s), 
occupation(s), and income(s), and other features of the student’s family. Demographic 
characteristics may be subject to special attention in education policy from local to national 
levels, but, with the exception of the student’s marital and parental status, are not subject to 
change. 

Demographic variables are normally considered in the context of other aspects of student 
experience, behaviors, and attitudes when attainment of any kind (e.g., high school graduation, 
test scores, grades, college degree) is the dependent variable. Indeed, that is the way this 
analysis treats demographic characteristics. But to demonstrate what happens to the 
demographic variables in the analysis, this section opens with a stark presentation. If someone 
asked us to explain bachelor’s degree completion for 1992 12th-graders who subsequently 
attended a four-year college at any time, and all we knew about them were demographics, what 
would the various associations of demography with degree attainment look like? 

Table 3 presents a logistic exploration that started with nine demographic variables described in 
Part II above. The socioeconomic status quintile variable is not included, but two of its 
components (family income and parents’ highest level of education) are present to serve as 
proxies for SES. Two of the nine demographic variables — whether the student was a nonnative 
speaker of English and family immigrant status — were not accepted into the logistic equation. 

Whether there were nine variables or seven, however, this logistic model fails to reach 
significance as a model, regardless of whether independent variables within the model turn out to 
be significant. The t value for the intercept falls below the threshold of 0.765 this study uses for 
keeping independent variables under consideration, and the proportion of concordant 
probabilities predicted at 63 percent is the lowest-by far-of any logistic model in this study. 

Of the independent variables within the model, four are significant, even though that significance 
is undercut by the statistical characteristics of the model as a whole. The most significant 
{p < 0.01) is first generation college status, with a Delta-p statistic that says the probability of 
completing a bachelor’s degree is reduced by roughly 21 percent for first generation students. 
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Race/ethnicity and gender are significant at p <. 05, with the messages of the Delta-p that 
minority status reduces the probability of earning a bachelor’s degree by 17 percent, and being 
male reduces that probability by 1 1 percent. Falling in the highest third of family income is 
marginally significant. 

Table 3. Logistic account of the relationships among major demographic variables 
and bachelor’s degree attainment for 1992 12th-graders who attended a 
four- year college at any time 



Adjusted 
Parameter standard 

Variable estimate error t p Delta-p 



Intercept 




0.2250 


0.1869 


0.69 






Race/ethnicity 




-0.7257 


0.1434 


2.90 


0.05 


-0.1660 


Gender 




-0.4760 


0.1072 


2.54 


0.05 


-0.1089 


Parenthood 




-1.6285 


0.5255 


1.77 


t 


t 


Siblings 




-0.3619 


0.1109 


1.87 


t 


t 


Family income 




0.3870 


0.1111 


2.00 


0.10 


0.0885 


First generation 




-0.9137 


0.1420 


3.69 


0.01 


-0.2089 


Urban high school 




0.1839 


0.1217 


0.87 


t 


t 



f Variable did not reach the minimum level of significance (p<. 1 0) in a two-tailed test. 

NOTES: Statistically significant variables are in bold. G 2 = 6288.09; df= 5144; G 2 /df= 1.222; X 2 (df) = 413.76(7); 
pseudo R 2 = .077. Proportion of concordant probabilities predicted = 63.0. Root design effect = 1.75. 

SOURCE: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402 and Supplement). 

But the most important message is the failure of the purely demographic analysis itself. If the 
statistical model does not reach a convincing level of significance, and all the variables inside 
the model are basically cut from the same cloth, then even if some of those variables are 
significant within the model, the cloth is frayed. That failure sets up a hypothesis that will 
unravel in the course of subsequent steps of the logistic narrative: Among students who attended 
a four-year college at any time, student demographics and family demographics may have, at 
best, indirect connections with degree completion. In fact, at the moment high school academic 
history is included in the multivariate account, demography plays a considerably reduced role at 
the same time that the logistic model becomes more persuasive. 

High School Background and History 

This is where the academic momentum story begins, where the major contributions of the 
original Tool Box were most prominent, and where the story continues to be of considerable 
importance. 
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There were four core high school background and performance variables in the first step of the 
logistic inquiry in the original Tool Box : senior year test score, a combined index of class rank 
and GPA, academic curriculum intensity, and the student’s education “anticipations.” The first 
three of these variables were combined in a composite index of “Academic Resources” that 
proved to have lasting positive repercussions through students’ postsecondary histories. 

Before we examine the results of the Step 1 logistic model, we should (a) revisit education 
"anticipations," and (b) provide a detailed account of how The Toolbox Revisited deals with high 
school curriculum. 

"Anticipations" versus "aspirations" 

Both the original Tool Box and this study employ a variable that some people will interpret as 
indicating students’ education "aspirations." But this construct is carefully labeled 
"anticipations," and is not based on the student’s answer to a question asked only once (for 
example, in a 12th-grade survey or in a freshman registration line). Rather, "anticipations" is 
built from sets of questions asked in both the 10th grade and the 12th grade, and describes the 
consistency and level of the student’s abstract expectations and concrete plans. 

Perhaps as a by-product of the national visibility of discussions on secondary-to-postsecondary 
transitions that followed the release of A Nation at Risk ini 983, along with a burst in recruitment 
of minority students by both colleges and professionally-oriented organizations such as the 
National Action Committee for Minorities in Engineering, the proportion of 12th-graders who 
consistently expected to earn at least a bachelor’s degree more than doubled from 22.5 percent 
(s.e. = 0.58) among 1982 12th-graders to 59.4 percent (s.e. = 1.02) among 1992 12th-graders. 
Continuing one’s education after high school, at some point and "in some form of postsecondary 
education" is now a norm of expectations among high school graduates: 92.6 percent (s.e. = 

0.54) of NELS:88/2000 students who graduated from high school with any kind of diploma, 
whether "on-time" in 1992 or later, expected to continue their education in a postsecondary 
setting. That which is a norm of behavior is like breathing in and breathing out. When virtually 
everyone expects to continue, the serious curricular “disconnect” between K-12 and 
postsecondary systems eloquently dissected by Venezia, Kirst, and Antonio (2003) is even more 
fraught with hazards and ironies. "Anticipations," one can surmise, will not play as significant a 
role in a logistic account of bachelor’s degree completion as it did in the original Tool Box. 21 

To aspire is an active concept. In our ordinary usage, it means to hope, seek, wish, or yearn. 
When Kao and Tienda (1998) describe "aspirations" as "a realistic evaluation of likely 
outcomes" (p. 352), they are not describing "aspirations" as we normally understand the term. 
Qian and Blair (1999), too, indicate that they use the "aspirations" to describe expectations or 
plans 



21 

Even there, the variable floated in and out of significance over the steps of the logistic model (see 
Adelman 1999, pp. 80-81). 
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"rather than a wish" (p. 622). And that’s just the point: With these rare exceptions, the literature 
on education "aspirations" consistently confounds an emotively-laden term applied to 
motivation, desire, or drive with an assessment of possibility based in experience (Mezias 1988). 
Underneath the consistency-by-level construct of the educational "anticipations" variable used in 
both the original Tool Box and in this study is what Kao and Tienda (1998) document as a 
growth from abstract attitudes about a very distant possibility when students are in junior high 
school to a rational judgment based on both school experience and input from parents and peers 
as the time draws closer to high school graduation. There is no doubt that the consistency-by- 
level formulation of our anticipations variable is influenced by what happens to students between 
grades 10 and 12. 

A different interpretation of expectations is implicit in Venezia, Kirst, and Antonio (2003): 
Expectations become assumptions, more passive than “aspirations.” Their principal thesis — that 
students falsely assume that the momentum inherent in a high school diploma is sufficient not 
merely to enter a postsecondary environment ("the easy part," in their words) but to complete a 
credential — is all the more powerful when these assumptions end in less than completion. The 
principal culprit lies in poorly articulated (secondary-to-postsecondary) curricula (Conley 2005), 
a topic addressed in Part VI of this essay. 

Academic Intensity of Secondary School Curriculum 

Think of a series of moving walkways in an airport. Passengers expect that one leads to the next, 
and don’t think too much about what’s happening along the way. What the most perceptive of 
contemporary analysts of the curriculum paths from high schools into colleges of any kind have 
observed (Venezia, Kirst and Antonio; and Conley) might be described as a disconnect in both 
the location and speed of these successive moving walkways, so that the passengers stepping 
mindlessly off the first and heading to the second trip, stumble into the moving hand rail, and 
desperately grasp at their luggage so that it doesn’t get stuck in the space between the two 
segments. This issue was imperfectly highlighted in the original Tool Box and is worth extended 
preliminary attention here. 

The “differential course work” hypothesis, originating in the work of Pallas and Alexander 
(1983) and Alexander and Pallas (1984) in relation to high school test scores, and validated by 
Answers in the Tool Box with a longer time frame and a bachelor’s degree attainment dependent 
variable, remains the core of the presentation in The Toolbox Revisited as well. With these 
forebears as guidance, we are going to look at the components of the curriculum variable, 
alternative presentations of those components for multivariate analyses, and some critiques of 
the measure. In the past decade, a number of short-hand representations of the high school 
curriculum, e.g., the New Basics as advocated in A Nation at Risk (1983), "minimum college- 
qualified" (Berkner and Chavez 1997), and "track location" (Lucas 1999; Lucas and Berends 
2002) have become common in both the research literature and the general press. The New 
Basics are the best known of these heuristics. The highest level of these "new basics" 
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— which simply counted Carnegie units 22 (four of English, three each of mathematics, science, 
and social studies, and two of foreign language) — has been labeled "rigorous" (McCormick 
1999), even though three units of mathematics could amount to no more than plane geometry 
and the science in question could lack any laboratory components. After the publication of the 
original Tool Box, the literature added criteria for highest level of math, core laboratory science, 
and Advanced Placement course work to the judgment of "rigor," and sought a trichotomous 
presentation with labels such as "core curriculum or less," "mid-level," and "rigorous" (Horn and 
Kojaku 2001). 

Quite frankly, the word "rigorous" is somewhat of a misnomer since a course requiring a high 
concentration of intellectual effort can be presented in a relaxed manner with comparatively low 
standards for success. Put another way, calculus or laboratory chemistry, for example, can be 
taught in a very laid-back fashion, while an otherwise "ordinary" survey of U.S. history can 
require the search for, discovery, and cataloguing of original source material, readings in 
archival methods, and frequent examinations and project presentations with criterion-referenced 
grading standards. Both the original Tool Box and The Toolbox Revisited prefer the term, 
"academic intensity," defend the continuous variable of academic curriculum intensity as a more 
flexible language of accounting, and decline to invoke labels of “college qualified,” “at risk,” 
and others. 

The academic curriculum intensity variable is called HSCURRQ. It takes a weighted 
distribution of NELS:88/2000 students across 31 levels of academic curriculum intensity and 
quality (there were 40 levels in the records of the High School Class of 1982, the subjects of the 
original Tool Box), and divides the distribution by quintile. Each level of academic curriculum 
intensity is a distinct configuration of numbers of Carnegie units 23 earned in core academic areas 
and other distinct notations about students’ course of study. For example, in both data sets, at 
the highest level, one finds students who, between grades 9 and 12, accumulated. 24 

3.75 or more Carnegie units of English; 

3.75 or more Carnegie units of mathematics; 

highest mathematics of either calculus, precalculus, or trigonometry; 

2.5 or more Carnegie units of science or more than 2.0 Carnegie units of core 
laboratory science (biology, chemistry, and physics); 
more than 2.0 Carnegie units of foreign languages; 
more than 2.0 Carnegie units of history and/or social studies; 
more than 1 Advanced Placement course; and 
no remedial English; no remedial mathematics. 



22 

““A Carnegie unit is the basic credit system for U.S. secondary schools. It is generally recognized as 
representing a full year (36-40 weeks) in a specific class meeting four or five times per week for 40-50 minutes per 
class session (Martinez and Bray 2002). 

23 

It is important to note that the Carnegie unit thresholds indicated, e.g., 3.75 units of English, are not 
determined a priori, but rather from the empirical distribution of Carnegie units standardized across the NELS high 
school transcripts in their edited version. See the descriptive windows for all the high school curriculum variables 
on the NCES data sets in CD # 2003-402 and its June 2004 Supplement. 

24 For mean numbers of Carnegie units earned in major curriculum areas, see Appendix L, table L6. 
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These are minimums. In fact, at the highest level of academic curricular intensity, students had 
accumulated an average of 4.30 Carnegie units of English, 4.34 units of mathematics (94 percent 
having reached precalculus or calculus), 3.63 units of core laboratory science, 3.76 units of 
foreign languages, and 3.79 units of history and other social studies, along with an average of 
three AP courses. That is an impressive portfolio. 

To this particular configuration, the NELS:88/2000 version of the highest level of academic 
curriculum intensity adds any Carnegie units in "computer science," a far more frequent and 
integrated elective in the high school curriculum by 1990, when the NELS students were in the 
10th grade, as opposed to 1980, when the High School & Beyond/So students were in the 10th 
grade (the mass availability and marketing of PCs began in 1984). 

Each subsequent level in a descending logical cascade subtracts something from this 
configuration (see Appendix F) until a relatively smooth distribution is achieved. Nothing is 
perfect in these algorithmic adjustments; and whatever lumpiness in distribution remains is 
assuaged in the quintile version. 

In light of recent literature on the effects of AP course-taking versus scores on AP examinations 
(e.g., Geiser and Santelices 2004; Klopfenstein and Thomas 2005), it is important to note that in 
both the original Tool Box and in this study, AP course work is not treated as a discrete variable, 
rather as part of the overall index of academic curricular intensity. However, in a moment we 
will demonstrate what happens when AP is invoked as a distinct independent variable in 
alternative constructions of the first step of our core logistic model. 

The basic critique of this account of high school curriculum has much merit: Numbers of courses 
(expressed in Carnegie units) in specific subject areas are not equivalent to high content 
standards or performance standards in those courses (Barth and Haycock 2004). In the 
cascading curriculum levels of the Academic Intensity variable, a student could accumulate three 
units of watered down history and social studies and still have three units. A national transcript 
accounting has no way of knowing what is demanded in thousands of history and social studies 
courses offered in 1100 high schools, a point made in the staff research for A Nation at Risk (see 
Adehnan 1983). The only domestic examples available for those who argue for test-driven high 
content standards in individual high school subjects such as U.S. history or chemistry or 
Algebra 2 are the College Board achievement tests (now called SAT II) and the New York State 
Regents. Some other states (e.g., Michigan) have tests in place for merit-based scholarships that 
cover broad areas of the curriculum (e.g., math, science) and language skills (reading, writing) 
but not specific subjects. In this respect, they are closer to the ACT examination, though at a 
less demanding level (Bishop 2004). The high school transcripts that serve as our evidence are 
very inconsistent in entering SAT II subject exam scores, so we have nothing on which to rely 
for an external validation of content. Getting beyond course titles to content standards is 
addressed in Part VI of this study. 

Altonji’s work (1996) offers a second important critique. He would have us control high school 
curriculum variables for school effects and "quality of courses," but admits that whatever one 
tries for proxy measures, the results would be "imperfect." Altonji draws our attention to the 
fact that more than half the variance in the mean number of Carnegie units earned in academic 
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subjects by the NLS-72 cohort 25 was across schools, indicating disparities in opportunity to 
learn, teacher quality, and student talents and proclivities. One might put Altonji’s conclusion 
more baldly: Students from higher SES backgrounds attend high schools that serve similar 
students, schools in which parental expectations and involvement are high, and schools in which 
curriculum offerings and teacher quality follow SES demands and can afford to do so because 
the tax base is higher. These students tend to enter college and earn degrees at higher rates than 
students from "lesser" backgrounds and from schools that accompany those backgrounds, 
therefore, the influence of the academic intensity of high school curriculum will be overstated. 

The implicit response of the original Tool Box made explicit in The Toolbox Revisited is that 
there are inequities in opportunity to learn and proclivity to learn that are reflected in the geo- 
demography of schools, but in a logistic account of bachelor’s degree attainment in which the 
results are reported with the Delta-p statistic, socioeconomic status will modulate some of the 
effects of curriculum. The message encourages all high schools to offer the requisite curricula, 
to make sure they have teachers who can deliver that curricula, to believe that their students can 
all reach higher levels of academic intensity in preparation, and to encourage their students to do 
so — no matter what students’ intentions for subsequent education or work may be. As the 
principles of the No Child Left Behind legislation are applied in high schools, this is a large part 
of what they mean. 

Highest level of mathematics and changes in high school course-taking 

In comparing the high school records of the HS&B/So (High School Class of 1982) with those of 
the NELS:88/200 (High School Class of 1992), and looking only at earned Carnegie units, what 
we principally witness is an improvement of precollegiate preparation of students in mathematics 

Table 4. Percentage of 1982 and 1992 12th-graders in academic high school programs 



whose high school curriculum fell below selected content and intensity thresholds 





Class of 1982 


Class of 1992 


Curriculum markers 


All 12th- 


In academic 


All 12th- 


In academic 


Percent of students with: 


graders 


program 


graders 


program 


Highest level of mathematics 
less than Algebra 2 


60.2 (0.78) 


35.4(1.02) 


37.7(1.11) 


24.2(1.24) 


Maximum of one year of core 
laboratory science 
Maximum of one year of foreign 


60.1 (0.82) 


37.1 (1.04) 


34.3 (1.17) 


21.0(1.22) 


language 


28.3 (0.88) 


14.3 (0.75) 


41.5 (1.30) 


28.4(1.47) 



NOTES: “Core laboratory science” is confined to biology, chemistry, and physics. Standard errors are in 
parentheses. Weighted N for students in academic program: HS&B/So = 1.5M; NELS:88/2000 = 1.8M. 
SOURCES: High School & Beyond/Sophomore Cohort (NCES 2000-194) and NELS:88/2000 Postsecondary 
Transcript Files (NCES 2003-402 and Supplement). 



25 

The National Longitudinal Study of the High School Class of 1972, the first, most elaborate, and longest 
(1972-86) of the NCES grade-cohort longitudinal studies 
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and science, a change that affects the amount of remediation necessary in four-year colleges, as 
well as mathematics course-taking at the college-level (Adelman, Daniel, and Berkovits 2003). 
We also note an unsettling increase in the percentage of students taking a maximum of one year 
of foreign language course work (though that percentage spread might be mitigated somewhat if 
we accounted for the increase in bilingual heritage language users between the two cohorts). 
Table 4 sets forth some basic threshold markers in these curricular areas. 

Thresholds are suggestive, but not conclusive, of the shape of change in curricular preparation. 
By reverse logic, table 4 says only that X percent of students are getting beyond point Q in 
secondary school study. It does not say how far beyond, and that may make more of a difference 
in academic momentum. The mathematics analysis in the original Tool Box was a proxy for the 
general metaphor of momentum, and it may be instructive to compare the results of those 
calculations — a distribution and a logistic — to what emerges from the NELS:88/2000 involving 
all the rungs on the “math ladder.” Tables 5, 6, and 7 do so, confining their universes only to 
students who were in the 12th grade in the year they were scheduled to be in the 12th grade. 

They do not restrict the universes to those who entered postsecondary education, let alone to 
those attended four-year colleges at some time. These are very inclusive groups. 

Table 5 confirms other observations in these data sets: The efforts following A Nation at Risk to 
move students further ahead in high school mathematics had positive results. Higher proportions 
of students were reaching Algebra 2 and higher levels. Bachelor’s degree attainment rates for 
those who moved at least one step beyond Algebra 2 in high school were stable, but declined at 
every level at or below Algebra 2. Table 5 provides a framework that needs to be validated in a 
more convincing way, and that is the purpose of table 7. Table 7 presents the results of logistic 
analyses and does so in odds ratios (departing from the presentation of other logistics in The 
Toolbox Revisited) to match the way in which this table was presented in the original Tool Box 
study. 26 

From the write-ups and presentations of the original Tool Box study a principle for high school 
curriculum guidance arose: "One step beyond Algebra 2 doubles the odds that you will earn a 
bachelor’s degree." What does that mean, i.e., how does one read table 7? An odds ratio 
indicates that every unit of change in an independent variable (in this case, each step up the math 
ladder) increases the odds of X happening versus the odds of X not happening by Y (the odds 
ratio). For table 7, the data for the full model — using only highest mathematics and 
socioeconomic status quintile — are presented first, in bold. In both cohorts, the odds ratio says 
that every step up the math ladder multiplies the odds of earning a bachelor’s degree by roughly 
2.5 versus an odds ratio for each step up the socioeconomic status quintile measure of 1.68 (for 
the class of 1982) and 1.78 (for the class of 1992). These data are statistically significant (p <.05 
or better). 



26 The High School Class of 1982 data in table 7 are slightly different from those in the previous 
presentation in Answers in the Tool Box (table 6, page 17) because a more accurate 12th-grade flag for that data set 
has been developed to match the NELS:88/2000. 
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Table 5. Bachelor’s degree attainment rate by highest level of mathematics reached in high 
school by 1982 and 1992 12th-graders 



Class of 1982 



Class of 1992 





Percentage 




Percentage 






reaching this 


Earned 


reaching this 


Earned 


Fevel of math 


level of math 


bachelor’s 


level of math 


bachelor’s 


Calculus 


5.2 (0.36) 


82.1 (2.45) 


9.7 (0.54) 


83.3 (2.72) 


Precalculus 


4.8 (0.37) 


75.9 (2.43) 


10.8 (0.65) 


74.6 (2.04) 


Trigonometry 


9.3 (0.51) 


64.7 (2.32) 


12.1 (0.81) 


60.0 (3.32) 


Algebra 2 


24.6 (0.75) 


46.4(1.54) 


30.0 (1.08) 


39.3 (2.31) 


Geometry 


16.3 (0.65) 


31.0(1.92) 


14.2 (0.87) 


16.7(1.87) 


Algebra 1 


21.8(0.69) 


13.4(1.33) 


16.5 (0.92) 


7.0(1.24) 


Pre-algebra 


18.0(0.66) 


5.4(1.19) 


6.7 (0.53) 


3.9(1.34) 



NOTES: Standard errors are in parentheses. The columns for level of math may not add to 100.0 percent due to 
rounding. 

SOURCES: National Center for Education Statistics: High School & Beyond/Sophomore Cohort (NCES 2000- 
194) and NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402 and Supplement). 

The balance of the table takes each level of high school mathematics, and runs a logistic 
regression for that level, with bachelor’s degree completion as the dependent variable, and SES 
quintile as the sole control. For the class of 1982, reaching calculus in high school increased the 
odds of earning a bachelor’s degree by a very impressive 8.18 to 1. For the class of 1992, the 
odds ratio for calculus was still in the same range at 7.52 to 1. The parameter estimates for the 
calculus line are almost identical (2.102 versus 2.018). These are consistent results. 

When one looks at the columns in table 7 for the class of 1982, one notices that the sign of the 
parameter estimate moves from negative to positive territory between geometry and Algebra 2, 
and the value of the parameter estimate rises above 1 .0 between Algebra 2 and trigonometry. 

The odds ratio more than doubles in each of those steps, but only in the step between Algebra 2 
and trigonometry (the "one step beyond") is the parameter also positive. For the class of 1992, 
all those relationships move up one rung on the "math ladder," principally because a higher 
percentage of this group (than the percentage for the class of 1982) reached precalculus or 
calculus while a lower percentage failed to get as far as Algebra 2. The critical boundary for 
math momentum now lies firmly beyond Algebra 2. 

But therein lies the rub, for not everyone has the chance to reach beyond Algebra 2. Differential 
lack of opportunity-to-learn was a major theme in the original Tool Box and is just as prominent 
a theme in this study of a cohort a decade later. Table 6 illustrates this unhappy situation. It 
asks what proportion of students — by race/ethnicity and socioeconomic status quintile — attended 
high schools that offered key math courses beyond Algebra 2. 
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If mathematics momentum is as important as we contend it to be, one can see the ripples of 
opportunity — or lack of opportunity — that start in high school offerings. In the matter of 
calculus, Latino students and those from any socioeconomic status quintile other than the highest 
are at a distinct disadvantage with respect to the opportunity even to confront the subject. 
African-American students join the other groups in less access to both trigonometry and 
statistics. While this issue will be revisited, it deserves underscoring here: The task of providing 
quality secondary school curricula to everybody, the paths to AP, the paths to the kind of 
learning challenges students will face in higher education, is enormous. If the promise of No 
Child Left Behind is to be realized at the secondary school level, it is first and foremost through 
the equitable provision of opportunity-to-learn. 

Table 6. Percentage of 1992 12th-graders who attended high schools that offered 
courses 3 in statistics, trigonometry, and calculus, by race/ethnicity and 
socioeconomic status quintile 

Percent attending high schools that offered: 



Demographic group 


Calculus 


Trigonometry 


Statistics 


Race/ethnicity 


White 


58.6(1.67) 


76.9 (1.29) 


27.7 (1.62) 


African-American 


50.8 (4.14) 


67.0 (3.90) 


19.5 (2.71) 


Latino 


44.6 (4.04) 


59.9 (3.55) 


18.2 (2.44) 


Asian 


61.3 (4.31) 


71.9(3.61) 


30.1 (3.94) 


Socioeonomic status quintile 


Highest quintile 


71.6(1.93) 


83.1 (1.64) 


34.0(2.30) 


2nd quintile 


56.2 (2.32) 


73.2 (2.13) 


27.1 (2.01) 


3rd quintile 


54.1 (2.39) 


71.4 (2.33) 


24.9(1.92) 


4th quintile 


49.3 (2.46) 


70.3 (2.28) 


20.3 (1.80) 


Lowest quintile 


43.5 (2.86) 


63.7 (2.66) 


18.5 (2.06) 



a Responses are based on surveys of school administrators and math teachers of NELS students in 1990. Where the 
administrator did not answer the question, the math teachers did not indicate that they taught the subject, and 
students did not earn any credits in the subject, the calculation assumes that the school did not offer the subject. 

This approach may underestimate the percentage of high school offering the subjects at issue. 

NOTES: Standard errors are in parentheses. 

SOURCES: National Center for Education Statistics: NELS:88/94 (NCES 96-130), and NELS:88/2000 
Postsecondary Transcript Files (NCES 2003-402). 

Since “highest high school mathematics” is a critical sorting component in the construction of 
the composite variable describing the academic intensity of a student’s secondary school 
curriculum, the ramifications of more students completing courses beyond Algebra 2, it is 
hypothesized, will ironically diminish (however modestly) the explanatory power of curriculum 
and raise the power of academic performance (class rank/GPA). To illustrate with a metaphor: If 
it was said that a student who reaches precalculus in high school and earns a C has a better 
chance of finishing a bachelor’s degree than a student who gets only as far as Algebra 2 with an 
A-, we now have to raise the bar for performance in precalculus to at least a B- in order for the 
prediction to hold. 
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Table 7. 



The math ladder for 1982 and 1992 12th-graders: Odds ratios and parameter 
estimates of earning a bachelor’s degree at each rung, controlling for 
socioeconomic status quintile 



1982 12th-graders 



1992 12th-graders 



Adjusted Adjusted 

Odds parameter standard Odds parameter standard 



Variable 


ratio 


estimate 


error 


P< 


ratio 


estimate 


error 


P< 


Highest math 


2.56 


0.938 


.0426 


.01 


2.50 


0.918 


.0467 


.01 


Socioeconomic 


status quintile 


1.68 


0.520 


.0359 


.02 


1.78 


0.571 


.0449 


.05 


By highest level 
of mathematics; 


Calculus 


8.18 


2.102 


.0426 


.01 


7.52 


2.018 


.1936 


.05 


Precalculus 


6.34 


1.846 


.1895 


.05 


4.91 


1.591 


.1592 


.05 


Trigonometry 


3.76 


1.352 


.1267 


.05 


3.05 


0.860 


.1422 


.10 


Algebra 2 


1.82 


0.599 


.0867 


.05 


0.89 


-0.115 


.1030 


t 


Geometry 


0.63 


-0.468 


.1108 


t 


0.24 


-1.434 


.1812 


.10 


Algebra 1 


0.19 


-1.666 


.1386 


.02 


0.07 


-2.688 


.2788 


.10 


Pre-algebra 


0.06 


-2.778 


.2614 


.05 


0.05 


-3.088 


.6684 


t 



| Variable did not reach the minimum level of significance (p < .10) in a two-tailed test. 

NOTES: For both 1982 and 1992 12th-graders, the universe consists of all students for whom highest mathematics 
in high school could be determined, SES data were available, and highest postsecondary credential at the end of the 
cohort study period was known. Root design effect for the high school class of 1982 (HS&B/So cohort)=1.59; root 
design effect for the high school class of 1992 (NELS:88/2000 cohort)=1.98. NELS:88/2000 goodness-of-fit data 
for the full model: G 2 = 8289.25; df= 9135; G 2 /.df= 0.91; percent concordant probabilities predicted = 84.3. 
HS&B/So cohort goodness-of-fit data for the full model: G 2 = 8371.74; df = 9288; G 2 /df= 0.892; percent 
concordant probabilities predicted = 80.0. 

SOURCES: High School & Beyond/Sophomore Cohort (NCES 2000-194); NELS:88/2000 Postsecondary 
Transcript Files (NCES 2003-402 and Supplement). 
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The general improvement in the high school curriculum profile of the NELS:88/2000 cohort is 
reflected, at least among students who subsequently attended a four-year college at any time, 
in diminished postsecondary remediation for that population. Table 8 sets out the distribution of 
what in the original Tool Box study was called the “remedial problem,” a logical cascade that 
distinguishes not only the amount of remediation a student takes in college, but also the kind of 
remediation at issue. The logical cascade begins with remedial reading, then sorts out those 
students whose only remediation consists of one or two courses in mathematics, then divides the 
balance of the remedial population between those who required more than two remedial courses 
and those who required only one. The type of remediation at issue for both these residual 
populations is usually writing, but also includes courses devoted to basic study skills and general 
developmental work. If the population included students who never attended a four-year 
college, this distribution would look very different. 27 

Table 8. Percentage of 1982 and 1992 12th-graders who attended a four-year college 
at any time, by type and amount of remedial courses taken in postsecondary 
education 





High School 


High School 


Type and amount of remediation 


Class of 1982 


Class of 1992 


Any remedial reading 
Remedial math only 


10.0(0.56) 


7.8 (0.72) 


(1-2 courses) 

More than 2 remedial 
courses of any kind 


14.0(0.65) 


9.8 (0.65) 


other than reading 
Only 1 remedial course 


14.5 (0.72) 


8.9 (0.64) 


other than reading 


9.0(0.50) 


6.5 (0.41) 


No remedial courses 


52.5 (1.04) 


66.9(1.07) 



NOTES: (1) Standard errors are in parentheses. (2) Columns may not add to 100.0 percent 
due to rounding. 

SOURCES: High School & Beyond/Sophomore Cohort (NCES 2000-194) and NELS:88/2000 
Postsecondary Transcript Files (2003-402). 



27 

“ Among all NELS:88/2000 students who participated in the 1992 survey and subsequently attended only 
community colleges, 64.5 percent (s.e. = 1.88) took at least one remedial course and 43.7 percent (s.e. = 1.98) took 
more than one. Among those who attended only sub-baccalaureate trade schools, which often do not require 
remediation, 32.4 percent (s.e. = 5.89) took at least one remedial course and 21.9 percent (s.e. = 6.07) took more 
than one. There is a considerable difference between the norms and requirements of community colleges versus 
those of other sub-baccalaureate institutions in the matter of remediation, and we should not mix the two in analyses 
of remediation. 
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Summary of high school curriculum and performance variables 

To remind the reader of our tools before heading into Step 1 of the logistic narrative: For the 
high school curriculum components of the analysis we have a composite variable called 
HSCURRQ, and four variables set forth in Part II above describing discrete components of 
HSCURRQ that we can test as proxy measures for the composite — highest level of mathematics, 
science momentum (a combination of highest math and core laboratory science credits), foreign 
language credits, and number of Advanced Placement courses. In a moment all these will appear 
in a correlation matrix that will provide a preview of how they might play out in multivariate 
analyses. 

For our other perfonnance measures, we bring forward from Part II a quintile presentation of the 
student’s high school class rank/GPA, and a quintile presentation of senior year test score. 

Academic Resources 

The original Tool Box advanced a notion explored and developed by Karl Alexander of Johns 
Hopkins and his associates in numerous exemplary contributions to the research literature: A 
student’s academic background was far more important than demographic variables such as 
gender, race/ethnicity, family composition, and socioeconomic status in relation to test 
performance (Alexander and Pallas 1984), entering higher education (Thomas, Alexander, and 
Eckland 1979), and, in one study, degree completion (Alexander, Riordan, Fennessey and Pallas 
1982). How does one reflect a composite idea of "academic background" of students coming out 
of high school and into postsecondary education? The key measure of Academic Resources (or 
ACRES, as the original Tool Box variable was called in order to elicit the idea of academic 
cultivation), combined curricular record, academic performance (class rank/GPA) as an indicator 
of student effort, and an external measure in the form of perfonnance on tests of general learned 
abilities. These three components, as we have seen, are set out in quintiles. To get a 
preliminary idea of their relative strength in relation to bachelor’s degree completion and to 
equalize the conditions of judging degree-completion rates by performance quintiles, table 9 
takes all on-time high school graduates in 1992 and indicates the percentage who completed a 
bachelor’s degree by December 2000 by quintile of each of the three components of Academic 
Resources. To be included in table 9, the student’s record had to contain positive values for all 
three components. 

Table 9 offers some hints of what multivariate analysis will confirm. Compared to the 
distributions for academic curriculum intensity and class rank/GPA, the test score quintile 
variable yields weaker degree completion rates in its highest two quintiles. This is a clear sign 
that the test score quintile will not have as strong an association with degree completion as the 
curriculum and perfonnance factors. At first glance, it looks as if class rank/GPA will be fairly 
dependable, but it has one minor bump. The reader is directed in table 9 to the percentage of 
eventual bachelor’s recipients from the lowest quintile of class rank/GPA (13 percent) compared 
to the percent of bachelor’s degree completers in the lowest quintile of test scores and academic 
curriculum (about 9 percent in both cases). The differences are statistically significant in a 
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descriptive account such as that of table 9, but how meaningful they turn out to be must await a 
multivariate context. In the meantime, one would posit that the class rank/GPA measure may not 
prove as strong as the curriculum measure. 



Table 9. Percentage of on-time 1992 high school graduates who continued their education 
in any postsecondary institution who completed bachelor’s degrees by December 
2000, by quintile performance in the three component variables of Academic 
Resources 



Percent completing bachelor’s degree 



Population Quintiles of Academic Resources components 



All on-time 1992 
high school graduates 
who continued to 
postsecondary 
education at any time 


Highest 


Second 


Third 


Fourth 


Lowest 


Curriculum 
Class rank/GPA 
Senior test score 


81.7(1.50) 
78.8 (1.47) 
74.9(1.72) 


60.5 (2.14) 
59.1 (2.24) 
53.9(1.92) 


35.5 (1.93) 
40.3 (2.11) 
37.2 (2.16) 


23.4(1.99) 

25.7(2.16) 

26.7(2.41) 


8.7(1.37) 

13.0(1.94) 

8.9(1.31) 



NOTES: Standard errors are in parentheses. Weighted N = 1.6M. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 



A multivariate analysis involving these variables, in fact, confirms our suspicions, and its results 
are presented in table 10. The model is the foundation for construction of the composite variable, 
Academic Resources, and is derived from ratios of the standardized beta estimates of the three 
high school performance indicators produced by a logistic regression with bachelor’s degree 
attainment as the dependent variable and no controls (the logistic regression itself can be found 
in Appendix L, table L7). The change in these ratios between those for the High School Class of 
1982 and those for the High School Class of 1992, while still leaving academic curriculum with 
the highest weight, narrows the gap between curriculum and performance, and lowers the 
strength of test scores as a component of Academic Resources. Given the differences in overall 
participation in the variable, i.e., the proportion of 12th-graders for whom all three components 
have positive values, some of the change may be an artifice of the NELS:88/2000 data set. But 
the principal message is one observed when commenting on the change in the distribution of 
students by highest level of mathematics studied in high school: The higher the average level of 
mathematics attainment, the more student effort, reflected in grades, will count. This theme will 
continue to play out when our story crosses the matriculation line into postsecondary education; 
and it was one of the principal points made by DesJardins, McCall, Ahlburg and Moye (2002) in 
their critique of the original Tool Box study. 
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When using all three measures — curriculum, class rank, and ACT test score — in a traditional 
linear analysis predicting first-year college grades, Pike and Saupe (2002) found curriculum to 
be the strongest, though their curriculum variable was dichotomous (students either meeting or 
not meeting the entrance specifications for a selective state university), which would naturally 
increase its power. Even when they introduced indirect effects of high school characteristics 
— control (private/public), size, mean ACT score of students, and mean proportion of students 
who attended the university — the parameter estimates for this curriculum variable were 
significantly greater than those for test scores or class rank. While The Toolbox Revisited is not 
in the business of predicting first-year grades, it is gratifying to note research that has similar 
respect for the propulsive power of course of study. 

Table 10. Component weights of the high school Academic Resources variable for 1982 
12th-graders and 1992 12th-graders who presented positive values for 
all three components 



Components of Academic Resources 


1982 12th-graders 


1992 12th-graders 


Academic curriculum intensity 


.41 


.42 


Classrank/GPA 


.30 


.33 


Senior test score 


.29 


.25 


NOTE: Columns will add to 1.00. 



SOURCES: High School & Beyond/Sophomore cohort (NCES 2000-194), and NELS:88/2000 Postsecondary 
Transcript Files (NCES 2003-402 and Supplement). 

A second message bears repeating from its formulation in the original Tool Box : Student effort in 
curriculum participation and performance over the three years of high school (grades 10 to 12) 
reflected in the transcript data is worth considerably more than perfonnance on a three-hour test 
on a Saturday morning. Test scores are a natural consequence of the academic intensity of 
curriculum and quality of student effort reflected in grades, and the weakening position of the 
test score variable in the ACRES configuration for the NELS:88/2000 cohort is a natural 
outcome of improvements in academic curriculum participation of the post-A Nation at Risk era. 

Correlations 

Now that we have all the precollegiate variables explained and in place, the process of bringing 
zoom and macro lenses of multivariate analysis to bear begins. The first stage sets forth a 
correlation matrix of precollegiate academic variables including three outcomes: on-time high 
school graduation, basic postsecondary “access,” and bachelor’s degree completion. Table 1 1 
presents the Pearson’s r results. Some of these correlations are weak, suggesting that one or both 
of the variables in question will not add to the explanatory power of a logistic regression. On- 
time high school graduation, for the most noted example, is not related to anything (this was also 
true in the parallel correlation matrix in the original Tool Box). Other relationships are not 
surprising, e.g., Advanced Placement has almost no bearing on entering postsecondary education 
(96.7 percent [s.e. = 1.48] of NELS 12th-graders with any AP courses entered the postsecondary 
sphere, versus 75.9 percent [s.e. = 1.01] of those with no AP course work). 
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Table 11. Correlations of major precollege Academic Resources variables and high school 

graduation status, college entry, and bachelor’s degree attainment by December 2000 
for 1992 12th-graders 



On-time 

Curriculum Highest Science Foreign Class Senior HS grad. Post- 
intensity AP math momentum language rank/GPA test standard secondary 

quintile courses (5 levels) (3 levels) (3 levels) quintile quintile diploma entry 

(CURRQ) (APCRS) (HMATH) (SCIMOM) (FLAN) (RANK) (TEST) (ONTIME) (PSENT) 



CURRQ 






0.368 


0.777 


0.774 


0.645 


0.572 


0.581 


0.167* 


0.352 


APCRS 








0.438 


0.366 


0.270 


0.326 


0.317 


0.050* 


0.111* 


HMATH 










0.869 


0.519 


0.618 


0.634 


0.124* 


0.301 


SCIMOM 












0.489 


0.594 


0.578 


0.118* 


0.279 


FLAN 














0.430 


0.510 


0.146* 


0.338 


RANK 














1 

I 

I 

I 


0.566 


0.185* 


0.298 


TEST 


















0.138* 


0.313 


ONTIME 




















0.168* 


PSENT 






















BACHELOR’S 




0.524 


0.319 


0.538 


0.530 


0.451 


0.493 


0.469 


0.129* 


0.332 



NOTE: All estimates except those noted with an asterisk are statistically significant at p<.05 or better. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402 and 
Supplement). 



True to its position in the construction of the composite academic curriculum intensity variable 
(abbreviated here as CURRQ), in fact, Advanced Placement does not exhibit as strong a 
correlation with the composite as do the mathematics, science momentum, and foreign language 
components. Why? Because AP is invoked as a sorting criterion in only six of those 3 1 levels of 
academic curriculum intensity (see Appendix F), while highest mathematics is invoked in 25 
levels, foreign languages in 18, and core laboratory science or all science in 30. Common sense 
says that we will find higher correlations of these other components with the composite 
curriculum variable. 

But among those other components, the correlation between the highest level of mathematics and 
science momentum (which includes highest math in its definition) is so strong (0.869) as to set 
off collinearity bells. The clear message is to use only one of them in any multivariate analysis. 
The highest level of mathematics was dropped because the variable represents only one 
curricular area, whereas SCIMOM carries two. Likewise, following Pedhazur’s (1982) rules of 
thumb for identifying potential collinearity problems from correlations, table 1 1 advises that if 
we use the high school academic curriculum index, we should not invoke either highest level of 
math or science momentum in the same multivariate model. 
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The only other observations of the correlation matrix of table 1 1 worth special attention are: 



• When bachelor’s degree completion is the reference point, the relative 
correlations of the curriculum, class rank/GPA, and test score variables are in 
roughly the same relationship observed in the more complex construction of the 
composite Academic Resources variable: Curriculum exhibits the strongest 
correlation, followed by class rank/GPA, followed by test score. 

• However, when the referent variable is simply entering postsecondary education, 
(a) the correlation coefficients are much lower, and (b) the test score variable is 
slightly stronger than class rank/GPA. 

The bottom line of these two conclusions: What counts for completion will be more potent than 
what counts for mere entrance (the "easy part," in the words of Venezia, Kirst, and Antonio, 
2003). 



Step 1: The First Logistics 

In the commentaries and critiques of these aspects of the original Tool Box, two potent questions 
were raised: 

1) Is a composite variable such as Academic Resources, a variable that yokes together 
very different types of measures (a basically qualitative curriculum index, class rank and 
GPA, test score), as convincing as these measures taken separately in any multivariate 
account of degree completion (DesJardins, McCall, Ahlburg, and Moye 2002)? 

2) Unlike class rank/GPA and senior test score, the academic curriculum intensity 
variable is built from components, some of which are high profile in their own right, e.g., 
Advanced Placement and highest level of mathematics reached in high school. If we 
substitute the major components for the composite in an account of degree completion, 
will they be as convincing? 

In light of these questions, three distinct ways of setting up the first step in the progression of 
logistic analyses that lead us to appreciate what makes a difference (and how much of a 
difference) in completing a bachelor’s degree for students who attended a four-year college at 
any time were explored. The first step covers both student demographics and high school 
performance. 

What happens to all those demographic variables in the "demography only" logistic of table 3 
when these three competing approaches were tried out? With the exception of gender, 
race/ethnicity, and becoming a parent by age 20, the minute the high school performance 
variables enter, the demographic variables disappear. Taken individually, none of them — first 
postsecondary generation status, second language dominant, recent immigrant status, family 
income, number of siblings, and urbanicity of community in which the student’s high school was 
located — meet the statistical criteria for either entering or staying in the logistic models 
employed in this study. But two of these variables, family income and level of parents’ 
education, play dominant roles in the construct of socioeconomic status, which itself more than 
qualifies for the logistic treatment. 
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Repetition of a special note on the components of the high school academic curriculum intensity 
variable is necessary in the context of this first step in the logistic narrative. With the exception 
of Klopfenstein and Thomas (2005), a spate of recent reports and commentaries on the Advanced 
Placement program (e.g., College Board 2005) claim that the original Tool Box demonstrated the 
unique power of AP course work in explaining bachelor’s degree completion. To put it gently, 
this is a misreading. AP is only a component of the academic curriculum intensity in both the 
original Tool Box and this replication. But to test the misreadings, AP was tried out as one of 
three variables that might serve as proxies for curriculum intensity. The other two variables 
tested as substitutes were a measure of momentum in science (SCIMOM) which includes 
elements of HIGHMATH and Carnegie units earned in both core laboratory science and all 
science courses, and number of Carnegie units earned in foreign languages. 

So, three distinct approaches to logistic regression models were tried out for Step 1 . The first, 
table 12, keeps the composite variable Academic Resources, and includes nothing else. The 
second divided Academic Resources into its three components. And the third dropped the high 
school academic curriculum intensity variable and replaced it with its three proxy components: 
science momentum, foreign language study, and AP. 

Which one of these three versions should be carried forward in the stepwise analysis? In the first 
version (table 12), maintaining the composite of Academic Resources, only Academic Resources 
and socioeconomic status quintile are statistically significant (though with a p value of <0.10, 
SES is a barely influential presence) The Delta-p statistic says that each step up the five-quintile 
ladder of Academic Resources improves the probability of degree completion by about 15 
percent. The Delta-p applied to socioeconomic status quintile says that for each step the 
probability of degree completion increases by about 7 percent. Expectations do not make a 
difference, and the demographic variables, while all carrying negative parameter estimates, do 
not meet the criteria for statistical significance either. Message: At this stage, and in this 
formulation, the composite representation of student learning in high school, Academic 
Resources, is a very persuasive construct. 

The second and third versions of the Step 1 logistic can be found in Appendix G, and both are far 
less convincing. In the second version, Academic Resources was divided into its three 
components: curriculum, perfonnance (class rank/GPA), and test score. The test score 
component, with a t statistic of 0.35 (far below the threshold requirement of 0.765), disqualifies 
this version from the logistic narrative. This result also reinforces our previous observation that 
when both academic curriculum participation and the quality of student effort within that 
curriculum (grades) improve, test scores will automatically rise and paradoxically become less 
important with reference to the dependent variable of degree completion. 

In the third version of Step 1, of the three proxy variables for high school academic curriculum 
intensity, only science momentum (that combination of highest level of math and number of 
Carnegie units in core laboratory science) emerged as statistically significant. Neither Advanced 
Placement nor foreign language study reached the threshold level of significance, and, with a 
t statistic of 0.75, foreign language study was on the borderline of disqualification in terms of 
further consideration. The senior test score quintile variable did not even meet the minimum 
statistical criterion for entrance. Had we taken this version forward into subsequent stages of 
student history we would have only one proxy for high school curriculum (science momentum), 
and no traces of the external measurement of general learned abilities that the senior year test 
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score quintile conveys. At least the composite index of high school performance, Academic 
Resources, includes a trace of external assessment. 



Table 12. Logistic account of factors associated with earning a bachelor’s degree in the 
history of 1992 12th-graders who attended a four-year college at any 
time — demographic and high school background, version 1: Using the 
composite variable for high school Academic Resources 



Adjusted 
Parameter standard 

Variable estimate error t p Delta-p 



Intercept 




-4.2762 


0.6360 


3.10 


.05 




Academic Resources quintile 




0.6439 


0.0662 


4.48 


0.01 


0.14924 


Socioeconomic status quintile 




0.2912 


0.0621 


2.16 


0.10 


0.06749 


Education expectations 




0.6272 


0.2065 


1.40 


t 


t 


Race/ethnicity 




-0.4093 


0.1941 


0.97 


t 


t 


Gender 




-0.4633 


0.1485 


1.44 


t 


t 


Parenthood 




-1.5757 


0.4790 


1.51 


t 


t 



f Variables did not meet threshold criterion for statistical significance. 

NOTES: Statistically significant variables are highlighted in bold. Standard errors adjusted by root design effect = 
2.17. G 2 = 5315.44; df= 4919; G 2 /df= 1.081; X 2 (df) =1074.9 (6); pseudo R 2 = 0.204; percent concordant predicted 
probabilities = 77.5. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 

Summary decision: The composite variable, Academic Resources, stays in the logistic 
narrative; other versions of representing high school performance fail. 
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Part IV 

Matriculation and Beyond: The Features of Postsecondary History 



On reflection, the original Tool Box made a mistake in the order and boundaries of the blocks of 
variables entered in its analysis. Following high school background, and tracking students who 
ultimately attended a four-year college at any time, it jumped first to financial aid, for which the 
High School & Beyond/Sophomore data set contained detailed infonnation (including 
unobtrusive Pell Grant files) covering the first four years of a student’s postsecondary history. It 
then considered a configuration of attendance patterns that covered everything from the timing 
of entry to postsecondary education to extended aspects of attendance such as whether the 
student ever left his or her first institution of enrollment and never returned to that institution. 
Thirdly, it followed this mixture of time periods with a block of first-year performance 
indicators. The order and content of these three blocks of variables scrambled time and student 
history. 

The Toolbox Revisited unscrambles that sequence so that momentum can be more clearly 
observed. We begin with the transition from secondary school to postsecondary matriculation, 
and focus on timing of entry and type of first institution. We then move on to first-year 
performance, the importance of which is fully justified by the research literature. Only after 
accounting for the first year do we confront financial aid (which, in the NELS:88/2000 data set, 
covers only the first two years of post-high school history, and is the weakest section of the 
study, by far). 28 Longer-tenn attendance patterns and extended performance (curriculum and 
trends in grades) come later, as they should. 

In shaping the universe for analysis so that a full history is visible, one minor change to the 
boundaries will be exercised. The NELS:88/2000 cohort was scheduled to graduate from high 
school in the spring of 1992. Not all of them did, so the upper boundary for high school 
graduation will now be set at December 1996. Very few cases are affected, 29 but, as the NELS 
history ends in December 2000, the point of changing the upper boundary is to allow enough 
time for late high school graduates to build a postsecondary history. 

In entering this sequence, the reader should note that the master program generating the mass of 
student-level variables included on the restricted NCES data sets of transcript-based files for 
both the HS&B/So cohort and the NELS:88/2000 was basically the same, hence, there were no 



28 

“ Financial aid comes after first-year performance because the story line is more interested in the potential 
relationship of aid to completion among those who are already enrolled. If the topic was initial access, then the 
financial aid package for the first year would be part of the postsecondary entry configuration of variables. The 
NELS:88/2000 unfortunately does not allow us to distinguish modes of financing postsecondary education in the 
first year from those of the second year. 

29 

Those who earned a high school diploma of any kind after December 1996 constituted 2.7 percent of all 
1992 12th-graders (s.e. = 0.16) and 0.7 percent of all 1992 12th-graders who subsequently attended a four-year 
college (s.e.= 0.16). 
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algorithmic differences that would account for change in student behaviors such as attendance 
patterns and academic performance. There are minor exceptions, and these will be noted in the 
narrative. 



Step 2: Matriculation 

The research literature has long held that where one starts out in higher education has enonnous 
consequences, particularly with reference to completing degrees (Velez 1985). The vast 
majority (78.7 percent; s.e. = 1.15) of the universe under study here started out in a four-year 
college. Do any characteristics of that first institution stand out in multivariate analysis? 

Only selectivity of the first institution was admitted to the Step 2 logistic model (the fact that the 
institution was a four-year college was not admitted). To be sure, there are other characteristics 
of the first institution of attendance. But if the dichotomous “selective” variable turns out to 
have but modest, if any, significance in the account, it is unlikely that other stock institutional 
characteristics — size, control, residential/commuter ratio — will have any influence, either. The 
best of the institutional characteristics variables in the literature is probably Stoecker, Pascarella 
and Wolfle’s (1988) “size.” This is not a simple measure, rather a factorial scale that includes 
total enrollment, student/faculty ratio, and public control. However attractive the concept, 
institutional size rarely breaks through as a stand-alone factor in literature with large national 
samples because of student taste — some like it large, some like it small — and taste is too variant. 
All this does not mean that where one starts out is irrelevant to completion, rather it directs our 
attention to other features of student academic history. More to the point, with 64.8 percent (s.e. 
= 1.06) of NELS:88/2000 students who attended a four-year college at some time attending more 
than one institution, and 26 percent (s.e. = 1.01) attending more than two, the task of ascribing 
influence to institutional characteristics is daunting (for a discussion of this issue, see pp. 81-84). 

In addition to the selectivity variable (SELECT), Step 2 includes NODELAY, a marker for direct 
entry to postsecondary education following high school graduation. It also includes a new 
variable, ACCELCRD, made possible by the construction of the NELS:88/2000 postsecondary 
transcript file to reflect the practice of dual-enrollment that expanded during the period the High 
School Class of 1992 was attending high school and that has become much more visible since 
then. ACCELCRD sums all college credits earned by course work prior to high school 
graduation, along with credits earned by examination — including AP, the College Level 
Examination Program (CLEP), and institutional challenge exams. Most of these credits were 
earned either prior to matriculation or during the first term of enrollment, though some were 
earned at later points in the student’s undergraduate career. A previous brief analysis of this 
phenomenon (Adehnan 2004a, pp. 55-56) suggested that acceleration might have a bearing on 
degree completion since the descriptive data indicated a positive relationship between the 
number of "acceleration" credits earned and both (a) high school academic curriculum intensity 
quintile and (b) selectivity of the first institution attended. 
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Table 13 sets forth the relationships of the variables in play at the postsecondary matriculation 
stage to bachelor’s degree completion for students who attended a four-year college at any time. 
Academic Resources is still in a commanding position, and the Delta-p statistic indicates that 
with each step up the quintiles of Academic Resources the probability of completion increases 
by 12.8 percent. Socioeconomic status quintile is still significant, though again, marginally so. 
Of all the new variables, no delay of entry alone is statistically significant, and its Delta-p says 
that students who enter college directly from high school increase the probability of bachelor’s 
degree attainment by 2 1 .2 percent, a very persuasive marker. 

Table 13. Logistic account of factors associated with earning a bachelor’s degree in the 
history of 1992 12th-graders who attended a four-year college at any time: 
Postsecondary entry phase 



Adjusted 
Parameter standard 

Variable estimate error t p Delta-p 



Intercept 


-4.2124 


0.6588 


2.02 


0.01 




Academic Resources quintile 


0.5541 


0.0715 


3.54 


0.01 


0.1283 


Socioeconomic status quintile 


0.2859 


0.0643 


2.03 


0.10 


0.0662 


Education expectations 


0.3462 


0.2032 


0.78 


t 


t 


No delay of entry 


0.9161 


0.2224 


1.88 


0.10 


0.2121 


Selectivity of first institution 


0.4470 


0.2301 


0.89 


t 


t 


Acceleration credits 


0.1904 


0.1196 


0.73 


t 


t 


Race 


-0.4709 


0.2130 


1.01 


t 


t 


Gender 


-0.4627 


0.1540 


1.37 


t 


t 


Parenthood 


-0.9639 


0.4597 


0.96 


t 


t 



1 Variables did not meet threshold criterion for statistical significance 

NOTES: Statistically significant variables are highlighted in bold. Standard errors adjusted by root design effect = 
2.19. G 2 = 5060.17; #=4913; G 2 /df= 1.030; X 2 (df) = 1101.0 (9); pseudo R 2 = 0.2127; percent concordant 
predicted probabilities = 78.5. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 

The selectivity of the first institution of attendance, while yielding a positive parameter estimate, 
does not reach the threshold of significance, and the t value for acceleration credits, at 0.73, falls 
just below the threshold for retention in the overall statistical model. The case of acceleration 
credits is one for which the author hoped for a better outcome, but once a rule is set, it is 
observed: The variable is dropped from subsequent steps. The demographic variables are 
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retained, but don’t tell us much. And education expectations barely stays under consideration 
with a t value of 0.78. Beattie’s human capital analysis (2002) downplayed education 
expectations as a central feature of explaining outcomes, particularly in consideration of group 
differences. It turns out that, even with a more sophisticated variable (our “anticipations”) than 
the customary way of marking “aspirations,” Beattie is right. 

Step 3: First-Year Performance 

Now that we have our universe of 1992 12th-graders in college, no matter where they started out, 
the most prominent initial marker of progress is a configuration of first-year academic 
performance indicators — and by "first year" is meant the first calendar year following the month 
in which the student first enrolled in their "true" first postsecondary institution after high school 
graduation. 30 

Most research on the critical first year of postsecondary education takes as its subjects students 
who start out in the fall term, and, by custom-and-usage, measures of retention (whether term-to- 
tenn or year-to-year) begin with fall enrollments. The transcript data, however, teach us that not 
all students commence postsecondary study in the fall term, and that there is a modest bias in 
drawing a universe based on fall tenn beginners only. For all 1992 12th- graders who entered 
postsecondary education at any time, 82.1 percent (s.e. = 0.83) started in the fall term, 5.8 
percent (s.e. = 0.46) started in the summer term, and 12.1 percent (s.e. = 0.71) began in the 
winter or spring terms (depending on local academic calendars). While the full data on first tenn 
of entrance are presented in Appendix L, tables L8a and L8b, it is important to note here that 
Latino and African-American students are less likely than white students to commence 
postsecondary study in the fall term, and the same is true when students from lower 
socioeconomic status quintiles are compared with students from the highest socioeconomic 
status quintile. Any measure of retention or completion that confines its universe to students 
who began their postsecondary careers in the fall term is, to put it gently, grossly incomplete. 

The cumulative number of variables qualifying for the logistic narrative increases by 50 percent 
when our field of vision opens out to first-year performance. The logistic model of the original 
Tool Box included three first-year performance variables, two of which were major contributors 
to understanding degree completion, and are carried forward: a marker for earning less than 20 
credits in the first calendar year and a tag for students whose first calendar year GPA fell in the 
top 40 percent of the GPA distribution for that period. The Toolbox Revisited adds dichotomous 
variables indicating whether the student took any remedial courses in the first calendar year, and 



30 

The “true” first institution of attendance excludes (1) colleges and community colleges in which the 
student was enrolled prior to high school graduation; (2) institutions in which the student was enrolled during the 
summer immediately following high school graduation and prior to fall term postsecondary entry (unless the 
institution was the same in both periods); and (3) “false starts,” that is, cases in which the student enrolled, but then 
withdrew during the first term of attendance, only to enroll and complete course work in a different institution at a 
later point in time (in these cases, the second institution is the “true first institution”). The true first date of 
attendance is the first month of enrollment at the true first institution. 
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whether the student earned any credits in college-level mathematics during that period. The 
mathematics variable, identified as potentially potent by Astin and Astin (1993), represents a 
major departure in the way The Toolbox Revisited approaches its subject, for it introduces the 
potential association of postsecondary curriculum with degree completion (the original Tool Box 
did not raise this issue). 

The first calendar year of attendance is the year in which students’ preparation for postsecondary 
education is most sorely tested (Pascarella and Terenzini 1991), and in which both remediation 
and college-level mathematics study serve as indicators of that preparation. The lower division 
curriculum sets out course "gateways" through which students must pass in both universities 
(Hanson 1998) and community colleges (Boughan 2001). Every school can identify its list from 
high enrollment courses 31 and there is no question that most of these gateways require a direct 
connection to the content of high school curricula that define "readiness." Remediation stalls 
student momentum toward those gateways; college-level mathematics itself is a gateway. 

In a different approach, American College Testing (2004) defines “college readiness” as a 
remediation-free postsecondary experience, and sets its benchmarks for “readiness” as 

. . . The level of achievement [based on ACT test scores] for students to have a 
high probability of success (a 75 percent chance of earning a course grade of C 
or better, a 50 percent chance of earning a B or better) in such credit-bearing 
college courses as English Composition, [College] Algebra, and Biology. . . (p. 1). 

With all due respect to ACT, earning a C — or even a B — in college algebra is not why most 
students go to school. There are better ways to benchmark, starting with the Academic 
Resources index of total high school perfonnance in which curriculum (including highest level 
of mathematics reached in secondary school) dominates and test scores do not. Instead of a 
grade in a single course, we then acknowledge the proportion of students who completed credits 
in any of five core college-level math courses during the first year of attendance, and, among 
those who became non-incidental students (i.e., earned at least an adjusted semester’s worth of 
credits), the proportion who ever completed credits in those courses. In table 14, one notes right 
away that among those who start in community colleges and complete college-level math 
courses in the first year, college algebra is the dominant math course. Among those who start in 
four-year colleges, it is not the dominant math course, even though it ranks at the top in terms of 
student participation. 32 



31 

Hanson’s (1998) list from the University of Texas at Austin ranges from Ecology, Evolution, and 
Society, to Introduction to U.S. and Texas Government, to General Psychology, to Calculus I and II for Science and 
Engineering Students. 

32 

“For more discrete details on credits earned in college-level mathematics, the reader is directed to table 
L9, Appendix L. 



47 




Ironically, the ACT study helps explain why curriculum quality emerges as more influential than 
test scores in building the Academic Resources composite, and justifies continuing to track 
curriculum (in this case, using mathematics momentum) across the matriculation line. The test 
scores are dependent variables that respond directly to increases in the concentration and level of 
study in each of the major secondary school content areas (see, e.g., Florida Department of 
Education 2005). The ACT study sees value-added in each step up the math curriculum ladder 
in terms of scores on the ACT math test. So when the dependent variable is degree completion, 
it is this curriculum — more than the test score — that sets the value-added tone. 

Table 14. Percentage of 1992 12th-graders who earned credits in five categories of college- 
level mathematics courses in the first calendar year of attendance, and among 
non-incidental students, 3 the percentage who ever earned credits in those five 



categories, by type of institution first attended 




Percent completing during 1 


Among all students earning 


College-level 


first calendar year 




more than 10 credits, percent 


math category 


of attendance, starting 


in: 


who ever earned credits in 






Community 






Four-vear colleges 


colleges 




College algebra 


21.7(1.01) 


17.0(1.19) 


28.2 (0.92) 


Precalculus 


18.6(0.85) 


4.9 (0.77) 


17.4 (0.69) 


Calculus 


17.6 (0.96) 


1.6 (0.26) 


16.8 (0.74) 


Finite mathematics 


4.6 (0.54) 


0.9 (0.30) 


6.4 (0.46) 


Statistics, probability 


5.3 (0.42) 


2.3 (0.58) 


19.6(0.82) 



a A non-incidental student is defined as one who earned more than 10 undergraduate credits. 

NOTES: Standard errors are in parentheses. Weighted Ns: four-year college beginners = 1.14M; community 
college beginners = 789k; all who earned more than 10 credits = 1.84M. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 

But does that mean that curriculum continues to have the same influence in a postsecondary 
setting? The issue comes up twice in the sequence of logistic models, first in table 15, which 
presents the results of the Step 3 logistic: first-year performance variables. What do we see? 

• Earning less than 20 credits in the first calendar year following postsecondary 
entry is a distinct drag on degree completion. The Delta-p says that falling below 
the 20-credit threshold lessens the probability of completing a bachelor’s degree 
by a third] 

• First-year grades, a proxy for both student effort and acclimation to the academic 
demands of a new environment (Pascarella and Terenzini 1991), move in the 
opposite direction from low credits: If one’s first-year GPA falls in the top two 
quintiles, the probability of earning a degree increases by nearly 22 percent. 
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Table 15. Logistic account of factors associated with earning a bachelor’s degree in the 
history of 1992 12th-graders who attended a four-year college at any time: 
First postsecondary year performance 



Adjusted 
Parameter standard 

Variable estimate error t v Delta-p 



Intercept 


-3.5834 


0.6054 


3.33 


0.01 




Academic Resources quintile 


0.3419 


0.0699 


2.75 


0.01 


0.0754 


Socioeconomic status quintile 


0.2879 


0.0569 


2.84 


0.01 


0.0635 


Education expectations 


0.4040 


0.1794 


1.27 


t 


t 


Selectivity of first institution 


0.4059 


0.1979 


1.15 


t 


t 


No delay of entry 


0.8153 


0.2779 


1.65 


t 


t 


Low credits in first year 


-1.5299 


0.1669 


5.15 


0.001 


-0.3372 


First- year grades 


0.9919 


0.1541 


3.62 


0.01 


0.2186 


College-level math in first year 


0.3603 


0.1479 


1.37 


t 


t 


Any first-year remediation 


0.4963 


0.1722 


1.62 


t 


t 


Race 


-0.3471 


0.1906 


1.02 


t 


t 


Gender 


-0.3414 


0.1372 


1.40 


t 


t 


Parenthood 


-1.0277 


0.3965 


1.46 


t 


t 



f Variables did not meet threshold criterion for statistical significance 

NOTES: Statistically significant variables are highlighted in bold. Standard errors adjusted by root design effect = 
1.78. G 2 = 4411.64; df= 4764; G 2 /df= 0.926; X 2 (df) = 1516.37(12); pseudo R 2 = 0.2893; percent concordant 
predicted probabilities = 83.3. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 

• Academic Resources and socioeconomic status are the only other statistically 
significant variables. The Delta-p story for socioeconomic status quintile remains 
at roughly the same level: Each step up the SES ladder increases the probability 
of degree completion for this population by 6 percent or so. The Delta-p strength 
for Academic Resources, however, falls from 12.8 percent in the postsecondary 
entry model to 7.5 percent here, a natural consequence of the introduction of other 
competing curricular and grade -based variables. 

• The two new curricular variables for remediation and college-level mathematics 
produce t statistics that justify carrying them forward, but do not reach even a 
threshold statistical significance level of p<0A0. 
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In the last observation there is a surprise that calls out for explanation. The parameter estimate 
for remediation in the first year is positive, not negative as conventional wisdom would assume. 
Students in this group who took any remedial courses in the first year earned bachelor’s degrees 
at a 48.7 percent rate (s.e. = 2.70) compared to 69.9 percent (s.e. = 1.26) of those who did not 
take remedial courses in the first year (not in table). That is a significant and meaningful 
difference from which, one might assume, remediation would be a negative. But there are other 
variables in this model that determine the sign and strength of the parameter estimate and its 
statistical significance. In a study of the academic careers of traditional-age community college 
students, the author found that remediation (one dichotomous variable marking any remedial 
reading, and one multi-level variable based on type and number of remedial courses) did not 
affect either transfer or (for students who did not transfer) completion of an associate degree 
(Adehnan 2005a). Other recent research, using a very sophisticated targeting of students whose 
need for remediation may differ according to the school they enter, goes beyond the finding that 
remediation is not a drag on degree completion to demonstrate that, in terms of persistence, 
remediation yields decidedly positive results (Bettinger and Long 2005). Two studies do not 
make for a definitive conclusion, but the evidence that students who successfully pass through 
remedial course work gain momentum toward degrees is beginning to build. 

As for the other variables in the Step 3 model, we note that NODELAY drops out of the 
statistically significant range with an indirect message of “once a student is in college, their 
performance counts more than when they arrived.” In other words, the ramifications of delayed 
entry can be overcome, but only with the kind of considerable effort reflected in first-year credit 
accumulation and first-year grades. The reader will note of table 16, however, that when 
financial aid variables are part of the logistic model, NODELAY rises back above the 
significance threshold. 



Step 4. Adding Financial Aid to the Equation 

There is a major difference between the High School & Beyond/So and NELS:88/2000 data sets 
in the nature and temporal coverage of financial aid information. For the HS&B/So, an 
unobtrusive Pell Grant file provided data on at least one type of grant-in-aid. Student reports 
covered four years (1982-86) of other types of grants-in-aid, loans, and work (including, but not 
confined to, college work-study from all sources) while enrolled. Three dichotomous variables 
resulted: GRANT, LOAN, and STUWORK. In the original Tool Box study, based on the 
HS&B/So data, GRANTS and STUWORK were modestly significant contributors to the 
explanation of bachelor’s degree completion in both the financial aid and attendance pattern 
steps of the model, but fell below the threshold of statistical significance as soon as first-year and 
extended performance factors were taken into consideration. 

The NELS:88/2000 financial aid infonnation covered the period 1992-94 only, i.e., the first two 
years following the modal high school graduation date for the cohort. In the initial input versions 
of the data in 1994 (the third NELS follow-up survey), the infonnation was provided by students 
for each institution attended, and in answer to the question, simply phrased, “besides your 
previous savings, how are you paying for this?” The options were grants, loans, work, parents, 
and other forms of aid. For students who attended more than one institution during the 1992-94 
period, it is thus possible for different combinations of financial aid types to be attached to the 
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discrete attendance cases. For our purposes, if a student reported using any one fonn of financial 
aid or support (grants, loans, work-study or campus job) at any institution attended, a positive 
entry was recorded for that type of financial aid or support for that student. 

Table 16: Logistic account of factors associated with earning a bachelor’s degree in the 

history of 1992 12th-graders who attended a four- year college at any time: 
Postsecondary financing 



Adjusted 
Parameter standard 

Variable estimate error t p Delta-p 



Intercept 




-3.5855 


0.6048 


3.33 


0.01 




Academic Resources quintile 




0.3362 


0.0701 


2.69 


0.02 


0.0751 


Socioeconomic status quintile 




0.2902 


0.0571 


2.86 


0.02 


0.0648 


Education expectations 




0.3993 


0.1788 


1.25 


t 


t 


No delay of entry 




0.7854 


0.1980 


2.23 


0.05 


0.1754 


Selectivity of first institution 




0.3962 


0.1980 


1.12 


t 


t 


First-year grades 




0.9878 


0.1541 


3.60 


0.01 


0.2207 


College-level math in first year 




0.3673 


0.1480 


1.39 


t 


t 


Low credits in first year 




-1.5148 


0.1674 


5.08 


0.001 


-0.3384 


Any first-year remediation 




0.4967 


0.1722 


1.62 


t 


t 


Work-study or campus job 




0.1785 


0.1528 


0.66 


t 


t 


Race 




-0.3504 


0.1906 


1.03 


t 


t 


Gender 




-0.3379 


0.1374 


1.38 


t 


t 


Parenthood 




-1.0293 


0.3959 


1.46 


t 


t 



f Variables did not meet threshold criterion for statistical significance 

NOTES: Statistically significant variables are highlighted in bold. Standard errors adjusted by root design effect = 
1.78. G 2 = 4396.88; d/=4763; G 2 /df= 0.923; X 2 (df) = 1519.14(13); pseudo R 2 = 0.2915; percent concordant 
predicted probabilities = 83.4. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 

For those who ask about the importance of initial financial aid package offerings to students’ 
choice of where to enter the postsecondary system, the NELS:88/2000 is of little help. The 
Beginning Postsecondary Students longitudinal study of 1995/96-2001, on the other hand, has 
the data elements but does not provide convincing guidance; For traditional-age students (i.e., 
those who began at age 20 or younger)in the BPS95/96-2001, 7 percent (s.e. = 0.7) of those who 
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started at four-year colleges and had applied to more than one school and 3.4 percent (s.e. = 1.6) 
of those who started at community colleges and had applied to more than one school said the 
financial aid package was the principal reason for their selection. 33 Such percentages do not 
merit further inquiry. 

We come to Step 4 of the logistic model, then, with a weak collection of dichotomous financial 
aid variables, and the results must be judged in that light. As table 16 reveals, only the variable 
indicating whether the student received College Work-Study funds from any source or held a 
campus job was even admitted to the Toolbox Revisited model, and even then, produced a t 
statistic of 0.66, below the 0.765 required for carrying forward to the next step. Before that next 
step is taken, though, this narrative needs an important pause. 

A Critical Pause for the Second Calendar Year 

The literature on postsecondary persistence, retention, and attrition has historically taken its 
pauses and measures at the end of the student’s first academic year. A more constructive notion 
for plotting the path of academic momentum on the far side of the postsecondary matriculation 
line may be to give students two full calendar years following the initial date of enrollment 
before taking their progress pulses, yet the research on this possibility has been surprisingly 
limited to date (see Smith 1995, and Nora, Barlow and Crisp 2005). 34 Looking backwards from 
the final degree completion status of our NELS:88/2000 universe to mark cumulative change to 
the end of the second calendar year following entry on key indicators provides a richer 
framework for advisement. 

"Retention" and its paradoxes 

No national database will support the assertion that a quarter of four-year college entrants and 
half of community college entrants "do not return for their second year" (Kazis 2004, p. 4). 
Contrary to this negative conventional wisdom, the proportion of students in the universe for this 
study who "persisted" to the second postsecondary year is substantial. The definition of 
"persistence" is active and student-centered, marks a calendar academic year as July 1 through 
the following June 30, and runs as follows: 

Whenever the student first enrolls and earns credits in postsecondary education (summer, 
fall, winter, spring) marks the first academic calendar year of their postsecondary history. 

If the student enrolls and earns credits at any time and at any institution during the next 
academic calendar year, that student has "persisted." 



33 National Center for Education Statistics, BPS95/96-2001 Data Analysis System; NCES 2003-173. 

34 Nora, Barlow, and Crisp (2005) offer a potentially promising analysis of a cohort at one nonselective 
four-year institution that is undercut somewhat by lack of data on age distribution and the fact that retention and six- 
year graduation rates are confined to that institution. But the authors include cumulative GPA to the second year, 
enrollment intensity, remediation, three core courses (what this study calls "gateways"), and a rough course 
withdrawal factor — all of which speak in favor of their approach. 
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"Retention," which puts the student in a passive role vis-a-vis a specific institution, is not the 
right word to describe this sequence: It’s "persistence." There are two important points to 
underscore: (1) This definition reveals an extremely high rate of persistence, but (2) whether we 
talk about institutional “retention” or student “persistence,” the true measure is not the fact of the 
event, rather the quality of the student’s record going forward. The second academic calendar 
year offers students the opportunity to recapture any lack of momentum of the first. In that 
respect, the second year may be even more important than the first. 

The unhappy paradoxes of retention rates and retention quality 35 are revealed in table 17. The 
universe for the table is confined to students with complete records. The proportion of 
students who enrolled sometime, somewhere in the second academic calendar year following 
entrance to postsecondary education is about 90 percent for all NELS:88/2000 postsecondary 
students and 96 percent for those with standard high school diplomas who subsequently attended 
a four-year college at some time (and in this group, just about everybody who started in 
community colleges). These data ought to destroy the mythologies of low first-to-second year 
retention (which, by the formulas most commonly applied, include only students who started 
full-time and in the fall semester and returned to the same school the following fall semester). 
This account gives students credit for persisting, and gives students who started in sub- 
baccalaureate trade schools credit for earning a certificate in their first year and then going on 
their way. 36 

However, this account also shows that, even when one confines the universe to the group being 
followed in The Toolbox Revisited, roughly one out of five entered the second year with low 
credit momentum, and roughly one out of six carried low first year GPAs. The overlap, too, is 
considerable: 40 percent of those coming out of their first year with fewer than 20 credits were 
also in the bottom quintile of first year GPA (not in table). Low credit momentum is also due to 
the number of courses taken in the first year, with 41.7 percent (s.e. = 2.46) of those who earned 
less than 20 credits attempting seven or fewer courses — a proxy for part-time status — versus 2 
percent (s.e. = 0.27) of those who emerged with 20 or more credits. The reason for counting 
course attempts and not credit attempts is a by-product of the volume of remedial courses (which 
do not carry additive credits). Half of the low first-year credit group were tied up in remediation, 
versus 17 percent of those who reached or exceeded the 20-credit threshold. 



35 

As Tinto (1987) noted, “The point of retention efforts is not merely that individuals be kept in college, 
but that they be retained so as to be further educated.” (p. 136) Poor retention quality significantly lowers the odds 
of further education. 

36 The persistence rates for all 1992 12th-graders noted in this paragraph are slightly different from those 
previously reported (Adelman 2004a, table 3.4, p. 42), where the weight for known postsecondary participants (not 
the weight for students with complete records) was used. 
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Table 17. Percentage of 1992 12th-graders with complete postsecondary records who 



persisted in postsecondary education from their first calendar year of 
enrollment to a second calendar year, by type of institution first attended, 
and, of those who persisted, percentage with lagging first-year performance 











Of those who persisted, 
first-year 

performance indicators: 


Student group 


Persisted 


Earned 

one-year 

certificate 


Did not 
persist 


Less than 
20 credits 


In lowest 

GPA 

quintile 


All 12th-graders 


89.7(0.57) 


0.9(0.13) 


9.4 (0.55) 


33.2(1.12) 


17.4 (0.81) 


Type of first 
institution 












Four-year college 95.2 (0.59) 
Community college 84.0 (1.12) 
Other sub-bacca- 71.5(3.06) 
laureate 


0.1 (0.03) 
0.4 (0.10) 
14.8 (2.52) 


4.7 (0.59) 
15.6(1.11) 
13.7(2.01) 


15.9(0.91) 
60.7(1.93) 
31.4 (5.17) 


15.2 (0.86) 
21.5 (1.76) 
11.9(2.70) 


All with standard 
high school diploma 
by December 1996 
who attended a 
four-year college 
at any time 


95.8 (0.50) 


0.1 (0.03) 


4.2 (0.50) 


21.9(0.98) 


15.5 (0.88) 


Type of first 
institution 












Four-year college 95.2 (0.59) 
Community college 97.9 (0.87) 
Other sub-bacca- Low N a 
laureate 


0.1 (0.03) 
2.1 (0.87) 
Low N a 


4.7 (0.59) 

# 

Low N a 


15.9(0.91) 
44.0 (2.93) 
Low N a 


15.2 (0.86) 
15.7(2.72) 
Low N a 



# Rounds to zero. 

“ Reporting standard not met. 

NOTES: Standard errors are in parentheses. Row totals for the three persistence/retention columns may not add to 
100.0 percent due to rounding. Weighted N for all 12th-graders with complete postsecondary records: 1.88M; for 
all 12th-graders with complete postsecondary records who attended a four-year college at any time and who earned 
a standard high school diploma by December 1996: 1.38M 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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Momentum at the end of the second calendar year 

First-year background certainly helps explain some of what transpires in student histories by the 
end of the second year. Tables 18 and 19 offer a skeleton of two-year performance so that future 
research on the momentum hypothesis can become more sophisticated than this outing. The 
information is presented in two separate tables to make sure the messages are clear. The most 
overwhelming phenomenon to note in table 1 8 is that by the end of the second calendar year, 
those who never earned their bachelor’s degree were already 25 credits behind those who did 
(57.4 credits earned by those who eventually received a bachelor’s degree versus 31.6 for those 
who did not earn the degree), and for those who were behind, there was no difference between 
starting in a four-year college and starting in a community college. One set of "culprits" was 
attendance pattern related: lower percentage full time, lower percentage continuously enrolled, 
lower percentage getting to and beyond 20 credits in the first calendar year. All these factors 
decrease the probability of completion. 

Credit momentum is clearly in play. For the universe of this study, the mean number of credits 
earned within two calendar years of the date of entry was 49.5 (s.e. = 0.5 1). It takes a lot of 
work in the second calendar year for those whose additive credit totals in the first year were less 
than 20 to reach that mean, and only 3.6 percent (s.e. = 1.03) did, compared with 77.6 percent 
(s.e. = 1.02) for those who earned 20 or more credits in the first year. 

But a second set of negative vectors is clearly that of academic performance. In table 18 the 
spread in mean grade point average between those who eventually earned the degree and those 
who did not is substantial by the end of the second year, and it is not surprising that nearly one in 
five of those who never earned any credential had already become status dropouts (see table 19). 
The data remind us of a common sense fact that few analysts or commentators ever acknowledge 
when writing about student academic careers after high school: A measurable proportion of 
postsecondary students will not perfonn well (for GPAs of status dropouts, by timing of 
pennanent departure from postsecondary education, see Appendix H, table H4). Higher 
education has standards, and some students will, in fact, be placed on academic probation or be 
formally dismissed for academic reasons, i.e., flu nk out. These compressed signals are not as 
likely to be entered on official records as graduation with honors ( cum laude, etc.). 37 



37 

Compressed signals are used to mark extremes of academic performance. The American Association of 
Collegiate Registrars and Admissions Officers has reported that 90 percent of its member institutions enter 
graduation with honors on transcripts (AACRAO 2002), and, in a 2005 survey of transcript practices , found that 70 
percent enter academic probation or dismissal (www.aacrao.org/pro_development/surveys/transcript05.htm). 
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Table 18. 



Of 1992 12th-graders who earned a standard high school diploma by 
December 1996 and attended a four-year college at any time, credits 
earned and GPA at the end of the second year following initial enrollment in 
postsecondary education, and extended postsecondary attendance and 
performance markers, by ultimate bachelor’s degree status 









Of those who attended a 


Selected attendance and 


Of those who 




four-year college, but did 


academic performance markers 


earned bachelor’s 


not earn bachelor’s 


Earned 20 or more credits in 


90.8 (0.79) 




46.8 (1.88) 




first year 
Always full-time 


79.3 (1.04) 




37.9 (1.85) 




Continuously enrolled 


93.5 (0.58) 




45.7(1.93) 




Cumulative credits and GPA 
two years from entry 


average 


mean 


average 


mean 




credits 


GPA 


credits 


GPA 


All students in universe 


57.4 (0.35) 


2.91 (0.14) 


31.6(0.86) 


2.13 (0.36) 


Started in four-year college 


58.8 (0.30) 


2.92 (0.14) 


32.1 (0.85) 


2.03 (0.39) 


Started in community college 
If no remedial courses in 


49.2(1.30) 


2.84 (0.42) 


31.6(2.20) 


2.37(0.81) 


first year 


58.8 (0.36) 


2.97 (0.14) 


34.1 (1.13) 


2.19(0.32) 



NOTES: Standard errors are in parentheses. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 



Table 19 serves another function in the narrative, namely, to look back from the end of the 
longitudinal study in December 2000, and ask what percentage of postsecondary students were 
no longer enrolled and had not earned a credential of any kind, by timing of the gap between 
their first month and last month of enrollment? These students are called "status dropouts" 
(though they may return to postsecondary education at a later date). While the general subject 
of timing and reasons for departure is elaborated in Appendix H, table 19 looks at the eventual 
dropout population only by their status at the end of the second year of attendance. It says, very 
simply that: 

(a) of those who never earned a credential, 23 percent became status dropouts by the end 
of the second year; 

(b) of all those who ultimately became status dropouts over an 8.5 year period, roughly a 
third did so by the end of the second year; 

(c) for those who dropped out in either the first or second year, GPAs were decidedly 
below thresholds required for degrees, but 

(d) when asked why they left without a degree, less than 6 percent cited academic reasons 
(also see Appendix H, table H2). 
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Yes, the status dropout rate proves higher during the first year than the second, but by the end of 
the second year, the loss of academic momentum is very evident. 



Table 19. Status dropout rates at the end of the second year following initial 

enrollment in postsecondary education of 1992 12th-graders who attended a 
four- year college at any time, and allied academic performance data 



Timing of departure 


Percent of those 
who never earned 
a degree who 
became status 
dronouts 


Percent of all 
who ultimately 
became status 
dropouts 


Mean grade 
point average 
at end of 
enrollment 


Percentage of 
dropouts 
who said 
they left for 
academic 
reasons 


Left postsecondary without 
credential, never returned: 










during first 1 1 months 


13.1 (1.62) 


19.1 (1.33) 


1.68 (0.81) 


5.9(1.92) 


during months 12-23 after 
first enrollment 10.3 (1 .26) 


13.3 (1.56) 


1.98 (0.68) 


5.6(1.40) 



NOTES: Standard errors are in parentheses. Weighted N for status dropouts in first two years = 103k. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 



Curriculum gateways 

As significant in the panorama of student performance through the end of the second calendar 
year are rates of participation in specific courses identified as curricular gateways in four-year 
colleges by Hanson (1998) and in community colleges by Boughan (2001). The taxonomy of 
postsecondary courses used to code the transcripts for both the High School & Beyond/ 
Sophomore cohort and the NELS:88/2000 data files includes over 1 100 discrete course 
categories 38 of which 645 were used to answer the following question: "What percentage of 
students in the Toolbox Revisited universe successfully completed credits in each course 
category between the month they first entered postsecondary education to a point two calendar 
years later?" Table 20 presents the 25 academic course categories 39 in which the highest 
percentage of those who eventually earned bachelor’s degrees garnered credits during that time 
span, and for each course category, also presents matching percentages for students who 
attended a four-year college at any time but who did not complete a bachelor’s degree. 



38 

The most recent version. Taxonomy of Postsecondary Courses Based on the National Transcript 
Samples, 2003, is available at www.ed.gov/rschstat/research/pubs/empircurr/index.html. 

39 

Though their enrollments were substantial, physical education activities, health information, physical 
conditioning, orientation, and personal development courses were excluded. 
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Table 20. 



Of 1992 12th-graders who earned a standard high school diploma by 
December 1996 and attended a four-year college at any time, course 
participation rates by the end of the second year following initial enrollment 
in postsecondary education, by ultimate degree status 



Percentage of students who earned credits by the end 
of the second year following initial enrollment 

Course Earned bachelor’s Did not earn bachelor’s 



English composition 


82.3 (1.03) 


53.4(1.40) 


General psychology 


61.5 (1.18) 


32.2 (1.19) 


General biology 


35.2 (1.24) 


12.4 (0.87) 


Introduction to sociology 


34.4(1.12) 


19.6(1.08) 


U.S. history surveys 


32.6 (1.22) 


14.9 (0.97) 


Micro/macroeconomics 


30.3 (1.14) 


9.3 (0.88) 


General chemistry 


30.1 (1.05) 


7.5 (0.74) 


College algebra 


26.7 (1.20) 


13.9(0.94) 


U.S. government 


25.3 (1.12) 


10.4 (0.74) 


Calculus 


23.7(1.11) 


3.2 (0.43) 


Precalculus 


22.4 (0.95) 


5.8(0.68) 


Oral communication 


20.4 (1.03) 


11.0(0.72) 


Introduction to philosophy 


18.9(1.05) 


5.0 (0.49) 


Literature: general 


18.9(1.03) 


5.0(0.56) 


Spanish: intro and intermed 


18.8 (1.01) 


5.9(0.61) 


Western civilization 


17.0 (0.93) 


6.5 (0.62) 


Introduction to computing 3 


15.8(0.90) 


10.9(0.81) 


Introductory accounting 


15.7 (0.81) 


7.2 (0.56) 


Statistics (mathematics) 


14.4 (0.79) 


3.7(0.68) 


World civilization 


12.1 (0.93) 


4.0(0.50) 


General physics 


12.1 (0.83) 


2.3 (0.42) 


Public speaking 


11.2 (0.78) 


6.2 (0.67) 


Music appreciation 


10.9 (0.84) 


3.8(0.45) 


Drama criticism/history 


10.7 (0.73) 


2.9 (0.46) 


American literature 


10.3 (0.71) 


1.7(0.28) 



a This is not "introduction to computer science." 

NOTES: Standard errors are in parentheses. Weighted N for bachelor’s recipients = 935k; for those who did not 
earn bachelor’s = 513k. All row estimate comparisons are significant at p<.05. 

SOURCE: National Center for Education Statistics: NELS:88/2000 postsecondary transcript files (NCES 2003-402 
and Supplement). 
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This is a story that moves curriculum details forward from high school. It is empirically-driven, 
which is to say that the volume of course participation identifies the gateways for both 
community college transfer students and those who started in four-year colleges in the 
NELS:88/2000 cohort. Given the disparity in the number of credits accumulated by the end of 
the second academic calendar year between those who eventually earned the bachelor’s degree 
and those who did not, the spreads in participation rates in these 25 course categories are not 
surprising. The ratios of these percentage participation rate contrasts are 7: 1 in calculus, 

6: 1 in American literature, 5: 1 in general physics, 4: 1 in general chemistry, and more than 3: 1 in 
precalculus, micro/macroeconomics, introduction to philosophy, general literature surveys, 
statistics, history/criticism of drama, and world civilization. These courses cover the full range 
of traditional lower-division distribution requirements (akin to pick one from bin A, two from 
bin C, etc.) in four-year colleges and general studies associate degree requirements (the bulk of 
the transfer curriculum) in community colleges, with the standard non-remedial English 
composition and college algebra the most visible gateways for the community college group 
(Boughan 2001). 

The list of 25 discrete categories in table 20 invites aggregation and, with aggregation, the 
inclusion of other lower division courses that serve gateway roles. For example, there are four 
college-level mathematics courses among the 25, and they certainly can be aggregated as we 
have already done in our assessment of first-year perfonnance. General biology is not the only 
introductory biology course category in the taxonomy used. Some institutions still use a 
zoology-botany sequence; others open the study of life forms with cellular biology; others, still, 
offer the option of human biology as a starting point. All of these course categories are in the 
taxonomy, and joining them with general biology in an aggregate allows a broader and more 
accurate coverage of participation in gateways to upper division course work requiring a 
grounding in the biological sciences. 

Table 21 takes this alternative approach to describing the curricular participation of students who 
earned bachelor’s degrees and those who did not. Table 21 uses 14 aggregate gateways, two of 
which are single-course categories repeated from table 20 (general psychology and 
micro/macroeconomics). While Appendix I provides a listing of all course categories under each 
aggregate, a few notes are necessary in the text: 

• Humanities other than literature covers multidisciplinary introductions to the 
humanities, introduction to philosophy, ethics, and introduction to religious 
studies (usually a comparative religions survey). It does not include foreign 
languages. 

• "Foundation business" is included for two reasons. First, business fields have 
historically claimed the largest percentage of majors among traditional-age 
bachelor’s degree completers and a substantial proportion of associate degrees 
awarded to traditional-age students by community colleges (Adehnan 2004a, 
tables 5.1, p. 61 and 5.4, p. 65). Secondly, prospective business majors appear to 
take at least one introductory gateway course in the area during their first two 
years (introduction to business, business law or business legal environment, 
and/or introductory accounting). 



59 




Table 21 displays not only the percentage of students in the bachelor’s and no-bachelor’s groups 
who earned credits in each of the 14 aggregates, but also, for those who earned any credits in the 
aggregates, the average number of credits earned. The reader will notice first, that the use of 
aggregates narrows the difference in participation rates between the two groups. For example, in 
the category of standard English composition, the difference in participation rates between those 
who eventually earned a bachelor’s degree and those who didn’t is 82.3 percent to 53.4 percent 
(table 20); but when the category becomes "college-level writing," and includes technical 
writing, creative writing, and advanced essay, the participation difference narrows to 84.5 
percent to 68.6 percent. That is still a significant gap, though not as severe as others in table 21. 
For the record, the credits earned in the aggregates of table 21 account for 59 percent of all 
credits earned by students in this study during their first two years of postsecondary education. 

Table 21. Of 1992 12th-graders who earned a standard high school diploma by 
December 1996 and attended a four- year college at any time, 
participation rates in lower-division course category aggregates and average 
number of credits earned in each aggregate by the end of the second year 
following enrollment in postsecondary education, by ultimate degree status 



Earned bachelor’s Did not earn bachelor’s 

degree by December 2000 degree by December 2000 





Percentage 


Average 


Percentage 


Average 




completing 


credits 


completing 


credits 


Course aggregate 3 


credits 


earned 


credits 


earned 


College-level writing 


84.5 (0.95) 


4.96 (.046) 


68.6 (2.05) 


4.83 (.091) 


Oral communication 


35.6(1.21) 


3.38 (.054) 


26.2(1.59) 


3.15 (.080) 


Computer-related 


24.5 (1.03) 


3.42 (.057) 


17.2(1.52) 


3.31 (.091) 


Intro biological sciences 


42.1 (1.25) 


5.21 (.088) 


22.3 (1.53) 


4.96 (.160) 


Intro physical sciences 


40.2 (1.15) 


7.46 (.142) 


15.8(1.33) 


5.79 (.223) 


College-level mathematics 


70.5 (1.20) 


6.30 (.103) 


37.5 (1.87) 


5.34 (.225) 


Core history 


56.0(1.27) 


3.04 (.132) 


34.6(1.82) 


4.13 (.099) 


General psychology 


61.5 (1.18) 


3.33 (.030) 


42.0(1.95) 


3.32 (.082) 


Micro/macroeconomics 


30.3 (1.14) 


4.69 (.088) 


13.1 (1.35) 


3.86 (.112) 


Humanities except literature 


38.2 (1.24) 


4.20 (.140) 


19.1 (1.50) 


3.55 (.124) 


Literature 


45.1 (1.30) 


4.48 (.087) 


19.8(1.39) 


3.84 (.144) 


Core social sciences 


62.6(1.27) 


4.57 (.080) 


42.8 (1.85) 


4.22 (.115) 


Visual/graphic arts 


17.3 (0.96) 


5.12 (.230) 


10.1 (0.98) 


5.47 (.488) 


Foundation business 


19.9(0.88) 


5.17 (.120) 


14.2(1.41) 


4.86 (.227) 



Tor a listing of courses under each aggregate, see Appendix I. 

NOTES: Standard errors are in parentheses. Weighted N for those who earned bachelor’s degrees: 935k; for those 
who did not earn bachelor’s degrees: 513k. 

SOURCE: National Center for Education Statistics: NELS:88/2000 postsecondary transcript files (NCES 2003-402 
and Supplement). 
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At the same time — and with few exceptions — the differences in average credits earned in the 14 
aggregates are negligible, statistically insignificant, or meaningless — or all three of the above. 

In other words, when students who did not earn a degree took courses in these aggregates, they 
earned credits at the same rate as those who did earn the degree. The problem is still that a much 
lower percentage of those who did not earn degrees even pass through these gateways. 

What does this account of the second calendar year mean? 

The sequence of steps of the logistic narrative of what matters in degree completion in 
The Toolbox Revisited moves from first-year perfonnance (with an apostrophe of financial aid 
considerations) to longer-term vistas of attendance patterns and extended perfonnance. It does 
not make a distinct stop at the end of the second year. And yet by the end of students’ second 
year, a significant spread in credit generation, academic performance, and curricular 
participation has opened up between those who eventually completed bachelor’s degrees and 
those who did not. An event history account appropriate to the analysis of withdrawal (e.g., 
DesJardins, Ahlburg, and McCall 1999) would include a measurement at this point. So as the 
logistic narrative heads into consideration of attendance patterns (Step 5), the reader might ask 
what would happen if we took first year-credits and first-year GPA and replaced those variables 
with second-year cumulative measures. 

An experimental logistic was constructed in response to that hypothetical question (see 
Appendix L, table L10). What happened? The Delta-p statistic for the credits variable barely 
budged and the Delta-p for GPA declined but slightly, i.e., at first glance, it doesn’t make a 
difference whether one uses first- or cumulative second-year measures. At the same time, 
though, the strength of that cumulative second-year credits substitute 40 wreaked havoc on critical 
variables that would otherwise be highlighted under attendance patterns, e.g., community college 
transfer and four-year-to-four-year transfer. As we will note in a moment, disentangling the 
consequences of different kinds of multi-institutional attendance is more critical to the analysis 
of degree completion than status at the end of the second year. 

What about the gateway lower-division courses taken by the end of the second calendar year? 
Are any of them key portals to degrees? Beyond college-level writing (a requirement) and 
college-level mathematics, this is not a fair question, for the answer depends on a student’s 
major and the requirements of the degree-granting department. Consider an English major who 
took general physics, the introductory micro/macroeconomics sequence, and introduction to 
design to satisfy distribution requirements. None of these courses (which are gateways to upper- 
division offerings in other fields) is a portal to completing a degree in English, though one could 
argue that they provide literature majors with analytic tools, reference points, and contexts that 
can only 



40 The variable for cumulative credits to the end of the second calendar academic year was constructed in a 
manner different from the dichotomous presentation of first calendar academic year credits. Cumulative credits is a 
categorical variable with four levels based on empirical mean credits earned and standard deviation (49 and 19 
respectively): 0-29, 30-49, 50-69, and 70 or more. 
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enrich their interpretive powers. In fact, it could also be argued that the distribution courses 
gathered in the 14 aggregates of table 21 are secondary in propulsive power to the second level 
of course work in specific fields. For example, "everybody" takes general psychology, but those 
who log experimental psychology and developmental psychology are establishing a beachhead 
in the field, and those courses become the real portals to the degree. 

One might use the number of curricular "gateways" through which students had passed by the 
end of their second calendar academic year as an independent variable, but the national samples 
with which we work in grade-cohort longitudinal studies would require consensus of academic 
administrators as to the curricular locations and expected threshold numbers of these gateways, 
and that consensus is simply not available. In fact, given the variety of presentations of 
postsecondary curricula across over 3,000 degree-granting institutions, that consensus may be 
impossible. This is ultimately a matter for local configuration and analysis (for a course cluster 
approach, see Zhai, Ronco, Feng, and Feiner 2001) as well for future research that would 
revisit the differential course work hypothesis in postsecondary contexts that appeared in the 
literature a decade ago (e.g., Ratcliff et al. 1995; Kroc, Howard, Hull, and Woodard 1997). As 
this pause in the narrative of The Toolbox Revisited demonstrates, though, those analyses cannot 
be profitably engaged on the basis of the student’s first-year work alone. The second year is just 
as important. 



Step 5: Postsecondary Attendance Patterns 

One of the principal contributions of Answers in the Tool Box was to document, on a national 
scale, what college and community college registrars, institutional research officers, and 
enrollment managers had known for some time: the increasingly complex enrollment patterns of 
postsecondary students (Hearn 1992; Kearney, Townsend, and Kearney 1995). This complexity 
has steadily accelerated since the 1970s, when the first high school cohort tracked by the 
National Center for Education Statistics attended college (for historical comparisons of multi- 
institutional attendance patterns, see Appendix L, table LI 1). It is not merely a case of the 
proportion of non-incidental 41 undergraduates attending more than one school increasing from 47 
to 57 percent; 42 it is more a question of how they attended more than one school, and in what 
combinations and order (Adehnan 2004a, pp. 45-50). 

The most critical distinction on this dynamic landscape is between transfer and multi-institutional 
attendance. Transfer is a migration that is formally recognized by system rules, a sequential 
movement from a de jure status in one institution to a de jure status at a second institution (or 
third, or fourth). Furthermore, the student’s stay at the second institution is not a short visit. So, 
for example, the beginning community college student who earns 46 credits at the first school 
and 5 credits at the second is a multi-institutional student not a transfer student. In the context of 



41 "Non-incidental" means students who earned more than 10 credits in their undergraduate careers. Those 
who earned 10 or fewer credits do not accumulate enough history for analysis of attendance patterns. 

42 Kuh et al. (2001) found that 53.2 percent of the 32 thousand seniors who participated in the 2000 
National Survey of Student Engagement had attended more than one institution. 
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a four-year student’s history, earning 102 credits at the first school (ultimately, the degree- 
granting school), 2 1 credits at a different four-year college and 9 credits at a community college 
along the way, involves a situation in which credits are transferred, not the student. But the 
student who begins at a community college and earns 44 credits at the community college before 
attending a four-year college, then earns 55 credits from the four-year college and is still enrolled 
at the four-year college at the end of our tracking period is a transfer student. 

Whether and how students attend more than one school may also depend on student performance 
in the first calendar year of enrollment. Using the five-year Beginning Postsecondary Students 
longitudinal study of 1989-94, McCormick (2003) demonstrated that for students who started in 
four-year institutions, the higher the first-year self-reported GPA, the more likely the student 
would stay put, whereas for students who started in two-year colleges, a higher GPA was 
associated with transfer, hence, with multi-institutional attendance. Do these findings hold up in 
a transcript-based longitudinal study such as the NELS: 8 8/2000? Table 22 outlines the 
corroboration: For four-year college students, McCormick’s observation is unconditionally 
seconded; for community college students, the difference between attending one school and two 
schools strongly supports the argument. 

Table 22. Relationships of grade point average (GPA) in the first year of attendance to 

number of institutions attended by 1992 12th-graders, by type of first institution 
attended 



Type and number of institutions 



First year GPA 



Started in four-year college 



One institution 2.74 (0.03) 

Two institutions 2.61 (0.04) 

Three or more institutions 2.53 (0.04) 

Started in community college 

One institution 2.39(0.05) 

Two institutions 2.59 (0.04) 

Three or more institutions 2.45 (0.09) 

NOTES: Standard errors are in parentheses. 



SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 



The students in the subject universe of this study were even more mobile than the 
NELS:88/2000 cohort as a whole: 35.9 percent (s.e. = 1.17) attended only one institution, 
compared with 46.2 percent (s.e. =1.04) of all NELS students who earned any postsecondary 
credits. And our subjects were more likely to attend more than two institutions than the larger 
universe (25.3 percent (s.e. = 1.15) versus 20 percent (s.e. = 0.81). This higher degree of 
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mobility invites us to identify the deeper structures of attendance so that we can better sort the 
relationships between starting point, performance, migration, and degree completion. 

For those students who attended more than one school, table 23 lays out some basic patterns. 

One should be very careful in reading them. The universe consists of the 64 percent of 1992 
12th-graders who not only attended a four-year school at any time but also attended more than 
one institution as undergraduates. Institutional mobility is built into the definition. It is not 
surprising (in fact, it is outrightly redundant) that reverse transfers do not earn bachelor’s degrees 
and that classic community college transfers who earn bachelor’s degrees receive them from an 
institution other than the one first entered. 

Those obvious observations aside, the most intriguing students in this configuration are what 
de los Santos and Wright (1990), Borden (2004), and others have called “swirlers.” The basic 
definition of “swirling” follows a pattern first observed in the High School & Beyond/ 
Sophomore cohort data and was labeled “alternating sectoral enrollment.” In this pattern, the 
student starts in either a four-year college or a community college, and moves back and forth 
between them for at least one cycle, accumulating more than 10 credits from both sectors in the 
process. In the sample isolated in table 23, 63 percent of these cycles began in four-year colleges 

Table 23. Combinations of institutions attended by 1992 12th-graders who attended a 
four- year college at any time who also attended more than one school, and 
percentage earning bachelor’s degrees under each combination 







Earned bachelor’s degree 


Institutional 


Percent 




From other than 


combination 


attending 


Total 


first institution 


Two or more four-year colleges 


31.8(1.41) 


82.3 (1.52) 


42.3 (2.57) 


Reverse transfer (four-year to 
community college) 


9.7(1.00) 


<0.1 (0.42) 


N.A. 


Community college to four-year 
college (including classic transfers) 


24.0 (1.37) 


58.1 (2.97) 


99.7 (0.26) 


Alternating sectoral enrollment 
("swirling") 


15.4(1.24) 


39.1 (3.77) 


69.7(4.55) 


Four-year-college-based student 
attending community colleges 
for incidental course work 


14.3 (1.01) 


86.4 (2.32) 


24.6 (4.39) 


Other combinations 


4.7 (0.62) 


4.6(1.42) 


45.1 (13.5) 



NOTES: Standard errors are in parentheses. Column for attending may not add to 100.0 percent due to rounding. 
Weighted N=753k. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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and 37 percent in community colleges (s. e. = 3.98; not in table). Furthermore, 57.6 percent of 
the students engaged in just one cycle of alternating enrollment while 42.4 percent rotated 
through two or more alternating cycles of attendance (s.e. = 4.05; not in table). Among the 
major categories of students who attended more than one school in table 23 there is a broad 
range of bachelor’s degree completion rates (39 to 86 percent), and the “swirlers” are at the low 
end of that range. 

What’s the point? Attendance patterns, as accounted for in the history of the High School Class 
of 1982 (the HS&B/So), presented paradoxes. As a dichotomous variable in the original Tool 
Box, multi-institutional attendance evidenced no association with bachelor’s degree attainment; 
classic community college transfer bore strong and positive fruit; but the more general 
phenomenon of attending more than one institution and not returning to the first institution of 
enrollment evidenced a strong negative relationship to degree completion. These paradoxes are 
unwound in table 24, the logistic account of the associations between the increasingly complex 
postsecondary attendance patterns of the 1990s and degree completion. 

As befits the topic of attendance patterns, the variables considered and admitted to the Step 5 
logistic model are those of time and place: 

Ever part-time: flagging those students who, at some time in their undergraduate 

careers, enrolled part-time in an academic tenn other than a 
summer term; 

Summer: number of credits earned during summer terms in three bands: 

none, 1-4, and more than 4; 

Classic transfer: community college to four-year after earning more than 10 

credits from the community college; 

Four-to-four: four-year college to four-year college transfer; and 

Multiple schools: marking students who attended more than one institution, whether 

they formally transferred or not. 

Actually, one major time -related attendance pattern variable is purposefully left out of the 
equation, continuous enrollment — or NOSTOP, as it is abbreviated. NOSTOP was one of the 
most powerful of the postsecondary history variables in the original Tool Box. Given the length 
of the grade-cohort postsecondary histories in the longitudinal studies, 43 a student was allowed a 
one-semester (or its equivalent, and exclusive of summer tenns) stop-out period and still 
considered continuously enrolled. When this variable is entered into the logistic model, its 
positive value and extraordinary statistical significance do not permit us to ascertain fully how 



43 Up to 12 years for the High School Class of 1972 (NLS-72), 1 1 years for the High School Class of 1982 
(HS&B/So), and 8.5 years for the High School Class of 1992 (NELS:88/2000). 
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Table 24. Logistic account of factors associated with earning a bachelor’s degree in the 
history of 1992 12th-graders who attended a four-year college at any time: 
Postsecondary attendance patterns 



Adjusted 
Parameter standard 

Variable estimate error t p Delta-p 



Intercept 




-4.6208 


0.7114 


3.68 


0.001 




Academic Resources quintile 




0.3648 


0.0773 


2.67 


0.02 


0.0804 


Socioeconomic status quintile 




0.2790 


0.0621 


2.55 


0.05 


0.0615 


Education expectations 




0.5165 


0.1985 


1.47 


t 


t 


No delay of entry 




0.9468 


0.3064 


1.75 


0.10 


0.2087 


Selectivity of first institution 




0.5176 


0.2155 


1.36 


t 


t 


First- year grades 




0.9295 


0.1687 


3.12 


0.01 


0.2049 


College math in first year 




0.3121 


0.1608 


1.10 


t 


t 


Any first-year remediation 




0.3261 


0.1876 


0.99 


t 


t 


Low credits in first year 




-1.1934 


0.1853 


3.65 


0.001 


-0.2712 


Classic community college transfer 




0.9518 


0.2252 


2.40 


0.05 


0.2097 


Four-to-four transfer 




0.7020 


0.2271 


1.75 


0.10 


0.1547 


Multiple schools 




-0.7509 


0.1908 


2.23 


0.05 


-0.1655 


Summer-term credits 3 




0.6517 


0.0866 


4.26 


0.001 


0.1436 


Ever part-time 




-1.6067 


0.1551 


5.87 


0.001 


-0.3545 


Race 




-0.3481 


0.2096 


0.94 


t 


t 


Gender 




-0.2955 


0.1498 


1.12 


t 


t 


Parenthood 




-0.8677 


0.4246 


1.16 


t 


t 



| Variables did not meet threshold criterion for statistical significance. 
a Set in three bands: 0, 1-4, and more than 4. 

NOTES: Statistically significant variables are highlighted in bold. Standard errors adjusted by root design effect = 
1.76. G 2 = 3749.31; df= 4759; G 2 /df= 0.788; X 2 (df) = 1984.37(17); pseudo R 2 = 0.3813; percent concordant 
predicted probabilities = 88.1. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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the other variables in the model are interacting. Its strength deflates the associative contribution 
of other factors in student performance that might better illuminate the dynamics of degree 
completion. Hence, in The Toolbox Revisited, we set aside NOSTOP until the last possible 
moment, the final Step 7. 

All five attendance pattern variables introduced in table 24 break the barriers of significance at 
this stage of model-building: 

• Ever part-time: Whether the student was ever part-time, a decidedly negative 
influence on degree completion. The Delta-p statistic tells us that part-time status 
reduces the probability of completion by over 35 percent. While previous 
research has documented the extent to which part-time enrollment pulls students 
off the "persistence track" (Carroll 1989), the 35 percent reduction in probability 
seems high given the fact that nearly half of the NELS: 8 8/2000 postsecondary 
students were part-time at some time. Students are inconsistent in their 
understanding of what part-time means, and under a more generous definition of 
the concept, the reduction in probability dropped to 30 percent — still a puzzlingly 
high number (see definition of PARTTIME in the Glossary). 

• Summer-term credits: The more, the better. That is, each step up the short ladder 
of summer-term credits increases the probability of completion by 14.4 percent. 

• Classic community college transfer: The significance of transfer is solid at 
p < .05, and the Delta-p says that the probability of completion for a transfer 
student increases by 2 1 percent. At one point in the construction of this variable 
for the NELS:88/2000 cohort, a flag was added to indicate whether the student’s 
first enrollment at the four-year college occurred within four years of entry to the 
community college. This attempted refinement of a time-sensitive variable did 
not change its position in the Step 5 logistic model. Given the fact that many 
students who ultimately transfer move back and forth between community 
colleges and four-year institutions, it is very difficult to detennine the precise 
moment of transfer, i.e., the season on the academic clock at which both 
institutions recognize a pennanent change in status for the student. A flag for 
first date of attendance at the four-year school does not help. 

• Four-year to four-year transfer: This variable is not as strong as community 
college transfer. Its statistical significance is marginal at p < .10. The Delta-p 
says that those who started in a four-year college and transferred to another four- 
year college increased the probability of earning a bachelor’s degree by 15 
percent. Some 18.5 percent (s.e. = 0.86) of those who started in four-year 
colleges transferred to another four-year college. In comparing those who 
transferred to those who didn’t, one could say only that the more selective the 
first institution attended, the less likely the student would transfer. 44 



44 0nly 5.4 percent (s.e. = 1.54) of NELS students who started in highly selective four-year colleges 
transferred to another four-year college, versus 15.7 percent (s.e. = 1.71) for those who began in selective 
institutions and 20.0 percent (s.e. = 0.99) for those who started in nonselective four-year schools. 
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• Multi-institutional attendance: In the steps of the logistic model of the original 

Tool Box 45 in which it appeared, the parameter estimates for this variable were 
-0.009, 0.015, and -0.077, and with odds ratios ranging from 0.93 to 1.02. All 
these data are basically flat and inconsequential. The situation in The Toolbox 
Revisited is different. In light of the two transfer variables and the ostensive 
intersection of multi-institutional attendance with transfer, what does this variable 
say by coming out of a logistic model with a Delta-p indicating that attendance at 
more than one school reduces the probability of earning a bachelor’s degree by 
about 16.5 percent? While further probing of the issue is advised, it appears that 
the message of the three variables (community college transfer, four-year to four- 
year transfer, and multi-institutional attendance) is that formal transfer is more 
effective than broader patterns of multi-institutional attendance, including 
simultaneous enrollment and “swirling.” This common sense encourages 
institutions to track and guide apparent nomads toward fonnal transfer. 

Of the other statistically significant variables in this step, even though Academic Resources and 
socioeconomic status quintile are receding in time, they are not so receding in strength, and are 
very much with the explanation of degree completion. In this new configuration, no delay of 
entry to postsecondary education and first-year grades are actually stronger than they were in 
Step 4 in their positive parameter estimates and Delta-p statistics. The negative consequences of 
earning fewer than 20 credits in the first calendar year following initial enrollment is also among 
the continuing statistically significant variables, though with a lower parameter estimate and 
Delta-p statistic than observed in tables 15 (first-year performance logistic) and 16 (financing 
logistic). What counts continues to count. 

And what counts less but remains statistically acceptable for presentation includes: (a) the set of 
basic demographic variables that has been in the logistic narrative from the beginning, 

(b) education expectations (which are proving far weaker for the High School Class of 1992 than 
they were for the High School Class of 1982), and (c) two curricular features of the first year of 
postsecondary attendance (remedial course work and credits in college-level mathematics) that 
are about to be reconstructed in Step 6, when we extend the temporal scope of curricular 
experience to a student’s entire undergraduate career. 

Step 6: Extended Postsecondary Performance 

Both the attendance pattern variables and those covered in this step (and the next) are 
distinguished from first-year performance considerations (Step 3) by the temporal reach of the 
observed behaviors. All the variables from Step 5 are carried forward. We then ask what 
additional key academic measures span the full range of a student’s undergraduate history — from 
entrance through the first calendar year of attendance, and beyond. 

As previously noted, consideration of continuous enrollment has been set aside for the last step 
in the logistic model sequence. Of other extended perfonnance variables, what was called the 
“DWI Index” in the original Tool Box and is described more prosaically here as the ratio of 
courses from which the student withdrew or repeated to all courses for which the student 



45 Adelman (1999), table 39, pp. 80-81. 
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enrolled, also is set aside for the last step, as it is another case of a very strong variable that 
would overwhelm other extended performance measures that the reader should see play out. 

The block of variables entered for the first time in Step 6 includes two reconstructions of 
curriculum-related factors originally presented within the confines of the first calendar year of 
attendance: type and extent of remedial problems (replacing first-year remediation), and 
cumulative credits in college-level mathematics (replacing first-year credits). “Remedial 
Problem” (see page 34 and table 8 above and description in the Glossary, p. 190) was not 
admitted to the model, while cumulative mathematics credits was easily admitted. 

The variable marking the trend in cumulative student grade point averages at three points in time 
is an improvement over its predecessor in the original Tool Box (which offered only two points 
of reference). Event-history analysts would no doubt want to see more points than three, but 
more points than three would result in the loss of more students than the 22 percent already lost 
by insisting on three (due principally to cases in which the true date of first attendance could not 
be determined). 

Table 25 offers a descriptive account of the three trend populations (rising, flat, and declining) in 
terms of average cumulative GPAs at each measurement point, the average elapsed under- 
graduate time period (in years) that is covered by those three markers, and the proportion of each 
group that earned a bachelor’s degree. The purpose of presenting table 25 is to demonstrate 
that the permutations of relationships between GPAs at the three moments of measurement work, 
and the results evidence prima facie sense. Students with rising GPAs are more likely to earn 
bachelor’s degrees than those in the other two groups, and students with falling GPAs seem more 
likely to spend more time as undergraduates (though the differences here are not great). The 
reader will also note that for the plurality of students, grades were flat over an average 
undergraduate stay of roughly four and three-quarter calendar years. 

Table 25. Three trends in postsecondary grade point average (GPA) of 1992 12th-graders 
who attended a four-year college at any time through December 2000 and 
offered complete postsecondary records, by GPA at three points in time, 
average undergraduate time, and percentage earning bachelor’s degree 



Average GPA 



GPA trend 


First 

calendar 

year 


First two 
calendar 
vears 


At end of 

undergraduate 

career 


Average 

elapsed 

undergrad 

time 


Percentage 

earning 

bachelor’s 

degree 


Percentage 
of all 
in group 


Rising 


2.43 (0.30) 


2.64 (0.28) 


2.93 (0.18) 


4.76 (0.57) 


73.5 (1.80) 


37.0 (1.09) 


Flat 


2.72 (0.25) 


2.63 (0.27) 


2.73 (0.26) 


4.79 (0.59) 


65.5 (1.63) 


43.9 (1.10) 


Falling 


3.09 10. 29) 


2.90 10. 29) 


2.70 (0.30) 


4.92(0.81) 


63.8 (2.38) 


19.1 (0.89) 



NOTES: Standard errors are in parentheses. Column for percent of all in group may not add to 100.0 percent due 
to rounding. Weighted Ns: rising GPA = 415k; flat GPA = 486k; falling GPA = 215k. 

SOURCE: National Center for Education Statistics: NELS:88\2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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Changing majors: Not a rare phenomenon 

St. John et al. (2004) complained that research on persistence pays inadequate attention to 
college major. Agreed. There are a number of problems with the extant research when major is 
an issue, not the least of which is the definition of “persistence” as within-year, i.e., fall to 
spring, and no attention to change- of-major. Stoecker, Pascarella and Wolfle (1988) paid 
considerable heed to major field, found some race-by-field interactions in a structural equation 
model, and did so within the context of a nine-year longitudinal study with degree completion as 
the dependent variable (a better framework than that used by others). But they faced problems of 
tracking change of major, moment of decision on major, the problem of students who drop out 
without a major, and the exclusion of community college transfers. 

In response to these difficulties, the Step 6 extended postsecondary performance model tried out 
a variable that marks change of major. This variable was not accepted into the model, 
principally as a by-product of overlapping characteristics with the community college transfer 
variable, as virtually all community college transfers “change major” from general studies to a 
specific field on entrance to a four-year college (see Glossary, p. 191). But it is worth noting 
that 50 percent (s.e. = 1.26) of NELS:88/2000 bachelor’s degree recipients changed majors at 
least once, and that this estimate comes very close to Simpson’s (1987) estimate of 48 percent 
for a 1976-84 cohort. The 50 percent mark thus appears to be a strong reference point. 

All the elements are now in place for the stage of analysis that examines student histories through 
the end of their undergraduate careers (with the exception of continuous enrollment and the ratio 
of withdrawals and repeats to all courses). The logistic for Step 6 is presented in table 26. 

There are three major observations: 

1) Race/ethnicity, never statistically significant in steps 1 through 5, does not even qualify 
for entry under extended postsecondary performance. Performance variables overwhelm 
demography. At this key inflection point in the analysis of degree completion, it appears 
race/ethnicity doesn’t matter (Light and Strayer 2002), though it may be operating as an 
indirect effect through other variables. But to test what happens in a different way, the 
Step 6 logistic model was run defining "minority" separately for each of the four major 
race/ethnicity groups. This test was run at this point in the logistic narrative because Step 
7 introduces two very powerful variables that would bury race/ethnicity indicators. The 
model itself emerged significant for white students, but not for African-American, 

Latino, or Asian students. For white students, all the significant variables in table 26 
were the same except for four-year to four-year transfer, which dropped below the 
threshold of significance. For Latino students, the only statistically significant variable 
within a nonsignificant model (t = 1.37 with 10 degrees of freedom) was whether the 
student had ever been part-time. For African-American students, the only statistically 
significant variables within a nonsignificant model (/ =1.70 with 13 degrees of freedom) 
were summer-term credits and whether the student had ever enrolled part-time. For 
Asian students, nothing was significant within a nonsignificant model ( t = 0.34 with 13 
degrees of freedom). 
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Table 26. Logistic account of factors associated with earning a bachelor’s degree in the 
history of 1992 12th-graders who attended a four-year college at any time: 
Extended postsecondary performance 



Adjusted 
Parameter standard 

Variable estimate error t p Delta-p 



Intercept 




-5.8188 


0.7996 


4.12 


0.001 




Academic Resources quintile 




0.3147 


0.0799 


2.23 


0.05 


0.0667 


Socioeconomic status quintile 




0.3066 


0.0628 


2.77 


0.02 


0.0650 


Education expectations 




0.3825 


0.2075 


1.04 


t 


t 


No delay of entry 




0.7798 


0.3208 


1.38 


t 


t 


Selectivity of first institution 




0.4103 


0.2225 


1.04 


t 


t 


Any first-year remediation 




0.2969 


0.1920 


0.88 


t 


t 


Low credits in first year 




-1.0822 


0.1957 


3.13 


0.01 


- 0.2294 


Classic transfer 




0.8391 


0.1273 


2.12 


0.05 


0.1779 


Four- year to four-year transfer 




0.7192 


0.2285 


1.78 


0.10 


0.1525 


Multiple schools 




-1.0523 


0.2005 


2.97 


0.01 


- 0.2231 


Summer-term credits 3 




0.5299 


0.0900 


3.34 


0.01 


0.1123 


Ever part-time 




-1.6696 


0.1599 


5.92 


0.001 


- 0.3539 


Cumulative college math credits 3 




0.5456 


0.0994 


3.11 


0.01 


0.1157 


Trend in grades 




0.5813 


0.1119 


2.94 


0.01 


0.1232 


First- year grades 




1.1619 


0.1860 


3.54 


0.01 


0.2463 


Gender 




-0.3518 


0.1578 


1.26 


t 


t 


Parenthood 




-0.9058 


0.4318 


1.19 


t 


t 



f Variables did not meet threshold criterion for statistical significance 
a Set in three bands: 0, 1-4, and more than 4. 

NOTES: Statistically significant variables are highlighted in bold. Standard errors adjusted by root design effect = 
1.76. G 2 = 3355.32; df= 4632; G 2 /df= 0.745; X 2 (df) = 1965.7 (18); pseudo R 2 = 0.3984; percent concordant 
predicted probabilities = 89.3. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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2) The new cumulative college-level math credits variable proves its worth, with a 
Delta-p statistic that says for every move up the credit-range the probability of earning a 
bachelor’s degree increases by 1 1.8 percent, putting it in the same range of influence as 
summer-term credits and trend in grades. 

3) When trend in grades is added to the variable block, the influence of first-year grades 
increases, in part because, by definition, a rising GPA trend requires a lower starting 
point. In fact, the first-year GPA quintile factor is twice as strong, measured in Delta-p, 
than the trend. 

We also note variables carried forward and remaining statistically significant in the model: 

• As predicted, Academic Resources continues to lose some of its power as students 
move further away in time from high school performance. There are too many 
intervening experiences. In Step 1, the Delta-p said that for each rung up the 
quintile Academic Resources scale, the probability of earning a bachelor’s degree 
increased by nearly 15 percent (see table 12 above). At this point, that increase in 
probability is 6.6 percent. Tested separately (see Appendix J), the tolerance 
collinearity statistic for Academic Resources dipped close to a 0.50 mark that was 
set for a caution light. While still acceptable, the tolerance statistic indicates that, 
at this point in a stepwise logistic with 17 other variables, a Venn diagram would 
show the circle for Academic Resources overlapping (but not eclipsing) many of 
the other variables in the model. 

• Socioeconomic status quintile continues to evidence some explanatory power, and 
with a Delta-p that basically doesn’t change from model to model. In this model, 
moving up each step of the SES quintile ladder increases the probability of 
earning a bachelor’s degree by 6.5 percent. SES would assume a more significant 
dynamic, as Cabrera, Burkum, and LaNasa (2005) have demonstrated with the 
HS&B/So cohort, if one entered the entire logistic narrative by SES quintile, and 
the elaboration of that dynamic will be left to future research. 

• The echoes of first-year credit generation are still noteworthy. The Delta-p says 
that dropping below 20 credits will decrease the probability of bachelor’s degree 
attainment by 22.4 percent. 

• Whether the student was ever part-time remains a very strong negative, with a 
Delta-p unchanged from its first appearance in the attendance pattern model. 

• Earning credits during summer terms is still good advice, though each step of that 
variable now adds 1 1.2 percent to the probability of earning a bachelor’s degree, 
as opposed to 14.4 percent when it was first introduced under attendance patterns. 

• Both the classic community college transfer variable and the four-year- to-four- 
year transfer variable stay in the model with positive contributions to the 
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probability of degree completion of 18 percent and 15 percent respectively, even 
as the multi-institutional variable strengthens with a negative parameter estimate 
and a restraining influence of 22 percent on the probability of degree completion. 
This tension continues to challenge analysis. 

The only other variable carried forward, but losing statistical significance is that indicating no 
delay of entry. No delay, which has bounced above and below the line of statistical significance, 
now drops below. 

Lastly, what has not counted in previous steps of the model still does not count: education 
anticipations, selectivity of first institution, first year remediation, gender, and parenthood. 

Three major themes emerge in this penultimate step of logistic analysis: 

• The combination of the strength of Academic Resources (composite high school 
performance, dominated by academic curriculum intensity) and cumulative 
college-level math credits underscores the power of curricular momentum on both 
sides of the matriculation line. 

• A group of attendance and perfonnance factors, including part-time status at some 
point, low credit momentum coming out of the first calendar year of enrollment, 
and multi-institutional attendance apparently colored by "swirling" and not formal 
transfer, are strong restraints on academic momentum. 

• Validating the event history critique of the original Tool Box offered by 
DesJardins, McCall, Ahlburg, and Moye (2002), two measures of GPA — first- 
year and trend — reflecting the quality of student effort, are very positive 
contributors to degree completion. The stronger position of the class rank/GPA 
quintile variable within Academic Resources reinforces this message. Grades 
count; and yes, there is a competitive message here, since our GPA scales and 
quintile breaks are relative, not absolute. 

Step 7: Final Factors 

The very last step in the logistic narrative takes all 17 variables from the extended postsecondary 
performance model of table 26, and adds the two strongest variables from the original Tool Box 
analysis. One of these, a dichotomous variable for continuous enrollment called NOSTOP, is a 
staple of analyses of postsecondary careers (e.g., Carroll 1989; Astin 1993; Horn 1998; Berkner, 
He, and Cataldi 2002). 

The second of these notable variables is not such a staple of analysis. It is the ratio of courses 
from which the student withdrew without penalty and those the student repeated to all courses in 
which the student enrolled. This is a rare topic in the higher education literature, and was last 
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seen in the major journals in Adams and Becker (1990), though the number of course 
withdrawals appears on the check list of elements in analyses of time-to-degree (Knight 2002, 
2004b). The ratio counts course attempts, not credits. Withdrawals without penalty are not the 
same as courses “dropped” within set periods most colleges and community colleges mark for 
“drop-and-add,” and dropped courses are not included in the ratio. As for no-credit repeats: Less 
than 4 percent of respondents to the 2002 survey of grading practices conducted by the American 
Association of Collegiate Registrars and Admissions Officers indicated their institutions did not 
allow repeats; fully 55 percent indicated that a student could repeat any course for a better grade; 
and 55 percent indicated that students could repeat a course as often as they liked (AACRAO 
2002). These proportions suggest a very lenient environment. Nothing here involves penalty 
grades. These are all cases of noncredit grades. The variable WRPT Ratio is dichotomous: On 
one side of the dividing line are students who withdrew from or repeated 20 percent or more of 
all courses for which they enrolled. 

Table 27 presents the final step of the logistic model for The Toolbox Revisited. The power of 
the last two variables entered is instantly obvious: 

• Withdrawing from or repeating 20 percent or more of courses decreases the 
probability of earning a bachelor’s degree by nearly half! ! ! 

• Remaining continuously enrolled increases the probability of degree completion 
by 43.4 percent. 

As a consequence of the introduction of NOSTOP and the WRPT Ratio, remediation in the first 
calendar year, a weakening variable from the point of its introduction, is forced out of the model. 
Its attempted replacement, the more elaborate description of type and intensity of undergraduate 
remedial problems (REMPROB), was tried out again, and again did not qualify to enter the 
model. As noted previously, remediation appears to be a neutral factor in this account. 

The marker for earning less than 20 credits in the first year of attendance, previously a strong 
contributor to the model, is turned on its head in the final step, and drops below the threshold of 
significance. Why? Principally because, as table 28 demonstrates, nearly 60 percent of those 
who wound up withdrawing from or repeating 20 percent or more of the courses for which they 
registered were already withdrawing from and repeating 20 percent or more of their courses in 
the first year. When one withdraws without penalty, one earns zero credits. When one repeats a 
course, one earns credits only once (assuming one passes, of course). What the relationships in 
table 28 strongly suggest is that low credit production in the first year is a logical consequence of 
withdrawal and repeat behavior. If we allow negative momentum to start early, the 
consequences will snowball. The phenomenon argues for more intense academic advising and 
monitoring, more accurate placement, and (in some cases) more sensible credit loads in the first 
calendar year of enrollment. 
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Table 27. 



Logistic account of factors associated with earning a bachelor’s degree in the 
history of 1992 12th-graders who attended a four-year college at any time: 
Final factors, with complete academic history 



Adjusted 
Parameter standard 

Variable estimate error t p Delta-p 



Intercept 




-7.6637 


0.8827 


4.89 


0.001 




Academic Resources quintile 




0.2766 


0.0847 


1.84 


0.10 


0.0583 


Socioeconomic status quintile 




0.2974 


0.0685 


2.45 


0.05 


0.0627 


Education anticipations 




0.4162 


0.2211 


1.06 


t 


t 


No delay of entry 




0.7848 


0.3515 


1.26 


t 


t 


Selectivity of first institution 




0.4436 


0.2432 


1.03 


t 


t 


First-year grades 




1.1020 


0.1119 


3.14 


0.01 


0.2323 


Low credits in first year 




-0.6553 


0.2165 


1.71 


t 


t 


Classic community college transfer 




0.7186 


0.2488 


1.63 


t 


t 


Four-to-four transfer 




0.6832 


0.2509 


1.53 


t 


t 


Multiple schools 




-0.7306 


0.2174 


1.89 


0.10 


-0.1540 


Summer-term credits 3 




0.5628 


0.0553 


3.25 


0.01 


0.1186 


Ever part-time 




-1.1739 


0.1009 


3.71 


0.01 


-0.2474 


Cumulative college math credits 3 




0.4993 


0.1075 


2.62 


0.02 


0.1053 


Trend in grades 




0.5879 


0.1211 


2.74 


0.02 


0.1240 


WRPT ratio 1 ’ 




-2.3078 


0.4246 


3.06 


0.01 


-0.4865 


Continuous enrollment 




2.0601 


0.2211 


5.25 


0.001 


0.4343 


Gender 




-0.3233 


0.1715 


1.06 


t 


t 


Parenthood 




-0.8511 


0.4627 


1.04 


t 


t 



f Variables did not meet threshold criterion for statistical significance 
a Set in three bands: 0, 1-4, and more than 4. 

b Ratio of withdrawal (W) and no-credit repeat (NCR) grades to all grades received. 

NOTES: Statistically significant variables are highlighted in bold. Standard errors adjusted by root design effect = 
1.76. G 2 = 2993.12; df = 4595; G 2 /df= 0.651; X 2 (df) =2260.53 (18); pseudo R 2 = 0.4382; Percent concordant 
predicted probabilities = 91.8. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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Lastly, the variable marking swirling multi-institutional attendance continues to be negative 
— though marginally significant with p <. 10 — in table 27 at the same time that both community 
college transfer and four-year-to-four-year transfer variables fall out of significance altogether. 
In this final accounting, then, all three type of multi-institutional accounts fade in associative 
power in the face of behaviors that transcend institutional effects: continuous enrollment and 
academic performance reflected in grades. 

Table 28. To what extent does the final ratio of undergraduate course withdrawal and no- 
credit repeat grades reflect the ratio and volume of withdrawal and no-credit 
repeat grades in the first calendar year of enrollment? Answers from the 
history of 1992 12th-graders who attended a four-year college at any time 



Final ratio of withdrawals 


Percent for whom 
first-year withdrawal 


Number of withdrawals and 
no-credit repeats in the first year 


and no-credit repeats to 
all courses attempted 


and repeat ratio was 
20 percent or higher 


None 


One 


Two or more 


Less than 20 percent 


5.4 (0.60) 


66.7(1.15) 


22.7 (1.06) 


10.6 (0.71) 


20 percent or higher 


57.5 (3.93) 


13.7(2.05) 


20.3 (3.17) 


66.0 (3.62) 



NOTES: Standard errors are in parentheses. Weighted N for those with a ratio of no-penalty withdrawals and no- 
credit repeats to all courses attempted of 20 percent or higher: 106k; of those with a ratio below 20 percent: 1.08M. 
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 



Summing Up the Logistic Narrative 

Parts III and IV of The Toolbox Revisited have walked the reader, sometimes painstakingly, 
through the consideration of independent variables for each of the seven roughly chronological 
steps from high school background to extended postsecondary performance. At the end of each 
step, a logistic account of what was associated with completion of bachelor’s degrees for 1992 
12th-graders who attended a four-year college at any time was offered. The logistic account 
applied to students’ histories as of that particular moment only. Step 7, of course, offered the 
concluding account. 

But to judge the full sweep of these seven steps, we ought to see them together in the same table. 
Table 29 allows summative statements about the changing strength of independent variables, the 
continuity of what counts, and required goodness-of-fit statistics for the whole model. Let’s start 
with the goodness-of-fit statistics because they validate the model. The reader can find them at 
the bottom of the second page of the table. Everything that is supposed to happen in a stepwise 
logistic model (Cabrera 1994; Menard 1995) happens. The G 2 (also called the "maximum 
likelihood function") falls with each step; G 2 /df (degrees of freedom in this calculation is the 
product of the unweighted number of students in the model minus the number of variables 
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in the model) also falls with each step, though perhaps too much in Step 7. The chi square 
statistic is a little more erratic, but rises as it is supposed to. So do the pseudo R 2 and the percent 
of concordant probabilities predicted at each step. These results are a relief to any analyst . 

Table 29 includes all the variables that were entered at any of the seven steps of the logistic 
account, even if they were not carried forward to subsequent steps because they did not meet the 
t statistic threshold criterion that was set at 0.765. In this manner, we can see where these 
variables were introduced and where they fell out of the model (the first "X" in the table marks 
the spot). Race/ethnicity is the most noted example because it was present from the beginning, 
and even though it was never statistically significant, it qualified to be retained until extended 
postsecondary performance variables were introduced. In Step 7 — and because race/ethnicity is 
a sensitive factor in public discourse on education — another approach was tried out whereby 
each of the four major race/ethnicity groups was treated as an independent variable in the model. 
Only the Asian student variable qualified for entry, but wound up with a t statistic less than 0.50. 

Another "casualty" — though for a very different reason — is the variable describing credits in 
college-level mathematics in the first calendar year. This variable was purposefully replaced in 
Step 6 (extended postsecondary perfonnance) by cumulative credits in college-level mathematics 
across the student’s entire undergraduate career because it was more appropriate to the temporal 
framework. First-year college-level math was never significant, but its more expansive 
replacement was, as might be expected from comparative participation rates in college-level 
mathematics by the end of the second calendar year of enrollment (table 21). 

Another virtue of the presentation in table 29 is that because all statistically significant variables 
are highlighted in bold, the reader can swiftly identify what was consistently meaningful among 
the potential associations with bachelor’s degree completion for this group of students. From the 
moment of their introduction, the following factors met the criteria: 

• Academic Resources quintile 

• Socioeconomic status quintile 

• Low (less than 20) credits in the first calendar year of attendance 

• First calendar year GPA 

• Summer-term credits 

• Ever part-time 

• Trend in GPA 

• Cumulative "career" credits in college-level mathematics 

And no delay of entry to postsecondary education from high school evidenced positive and 
significant association with degree completion in half the stages of the logistic narrative in which 
it was in play. 
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Table 29. Seven steps of a logistic regression model with bachelor’s degree attainment by age 26 or 
for 1992 12th-graders who attended a four-vear college at any time 



Attendance 

*' 1 ' " ‘ Patterns 



Background 



Entry 



First Year 



Financin 



27 as the outcome 



Extended 

Performance 



Final 

Factors 



Race/ethnicity 



Gender 



Parenthood 




First institution 
was selective 



No delay entry 



Acceleration 



Low credits 



First-year 

grades 



First-year 

remediation 



First-year 
college math 




















































































































































a Ratio of withdrawal (W) and no-credit repeat (NCR) grades to all grades received. 

NOTES: Keys to significance levels: ~ = .10; * = .05; ** = .02; *** = .01; + = .001. X = variable 
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files 



did not meet criterion to be carried forward. 
(NCES 2003-402 and Supplement). 
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When one adds the withdrawal/repeat ratio and continuous enrollment (which appear only in the 
last step of model-building), some rough themes emerge. Of the ten independent variables, three 
involve uses of time (continuous enrollment, ever part-time, summer-term credits); and four 
clearly reflect different aspects of academic performance (high school Academic Resources, 
first-year GPA, trend in grades, low-credits in the first year, and the withdrawal/repeat ratio). 

Under the temporal theme, no delay of entry floats in and out of statistical significance. Some 
might say, e.g., of no delay of entry, that the fact a variable is significant in one step of the 
logistic and not in another "does not always mean that different effects are operating" (Anaya 
1999, p. 507). One of the reasons that the logistic narrative provides collinearity statistics (see 
Appendix J) is to reassure the reader that, for whatever reason a variable ducks in and out of 
significance, it is not because it is excessively tangled up with other variables. The "tolerance" 
collinearity statistic for no delay of entry is consistently in the high comfort zone. 

Summary of Parts III and IV: Investment Behaviors 

When we look across the series of ingredients that rise to the top of strength of association with 
degree completion, we can interpret them in terms of what Noxel and Katunich (1998) called an 
"investment model," though this author prefers "investment behaviors," a microscale version of 
human capital. Once the modest consequences of socioeconomic status are accounted for, each 
step offers students a set of decisions that require the commitment of time and effort likely to 
yield a future benefit. These decisions move students, sometimes smoothly and sometimes less 
so, toward the degree. The choices made, beginning with high school curriculum (from the 
available curriculum — which is an opportunity-to-leam issue) and quality of effort in high school 
(reflected in class rank/GPA), allow subsequent leverage. Entering a postsecondary institution 
directly from high school, earning 20 or more credits in the first calendar year of enrollment, and 
performing well enough in that first calendar year to fall in the top 40 percent of a GPA 
distribution build on previous investment, and are all signs of commitment. Subsequent choices 
that may not be reflected in a bounded period of time, such as excessive course withdrawals, 
prove to be poor decisions with negative returns, breaking accumulated momentum. Other 
configurations of choice, including summer-term credit generation, meeting the challenge of 
college-level mathematics, effort that yields a rising GPA, and most of all, remaining 
continuously enrolled, all reflect continuing leverage of attainment. This is what academic 
momentum is all about. 

Tinto’s (1987) approach is just as direct: Students have responsibilities, and are expected to 
invest time on behalf of their own learning. Yes, in his words, students have the right “to refuse 
education” (p. 135), but since the primary commerce of the institution is education, those who 
refuse should not be surprised if (in his very delicate phrasing) the institution exercises its “right 
. . . to be selective in its judgments as to who should be further educated” (p. 135). One begins 
to see why student choice (and the responsibility inherent in student choice) emerges not only as 
a dominant theme of The Toolbox Revisited as well, but as the principal challenge to academic 
advising and counseling from secondary through postsecondary education. 



80 




One grants that a great deal more is going on in the lives of traditional-age students at the same 
time. They are not bloodless. The literature on persistence has included not only a mass of 
psychological and social considerations (Harmston 2004; Hu and St. John 2001; Kahn and Nauta 
2001; Robbins et al. 2004), but also basic aspects of maturation such as physical fitness (Zhang 
and Richards 1998), nutrition, roommate conflicts, dating problems, time management (Purcell 
and Clark 2002), and personality-environment congruences and dissonances (Feldman, Smart, 
and Ethington 2004). Studies such as Stage’s investigation of “motivational orientation” (1989) 
are also valuable in that they override stock demographic explanations and identify “subsets of 
college students who might react similarly to college experiences” and, hence, assist advisers 
and student service personnel. Our data sources do not include these factors. 

What the Literature Calls “Institutional Effects” 

A major strand of the literature on postsecondary retention derives from Tinto’s (1987) 
constructs of academic and social integration, constructs that contributed to the emergence of the 
National Survey of Student Engagement (Kuh 2001) and its use by hundreds of institutions in 
assessing the effects of environments on student behaviors. The reason Tinto’s work generated 
such a continuing elaboration and response is a combination of its elegance and prima facie 
common sense. At whatever age they start out, entering postsecondary students are not empty 
vessels; they come with demographic characteristics and high school experiences (and, if there is 
a gap of years between high school graduation and postsecondary entry, work experiences and 
family formation as well) that condition and shade where and how they enter the postsecondary 
system. Once at an institution, these background characteristics interact with the academic 
processes and social environment of that institution to yield varying degrees of determination to 
persist and complete credentials. Institutional culture, including habits of faculty interactions 
with students outside of the fonnal classroom and opportunities for a variety of peer group 
interactions, plays a significant role in Tinto’s models of academic and social integration. 

It is not surprising that the mass of studies drawing on Tinto’s work are institution-specific or 
use institutional characteristics as independent variables. To attain enough depth on all these 
features of student lives so that direct and indirect influences can be mapped through path 
analysis requires extraordinary probing, which is best carried out in institutional contexts where 
adequate samples can be assembled. Even if we employed hierarchical linear models to 
illuminate the relationships between nested characteristics of individuals within institutions and 
outcomes such as GPA, we would need much larger samples of students within each institution 
than the NELS:88/2000 provides. 46 



46 In a case of a continuous dependent variable for student engagement, and sets of independent variables 
describing both student and institutional characteristics (half of them continuous), Porter (2005) demonstrates the 
statistical superiority of multilevel analysis versus Ordinary Least Squares analysis. There are three significant 
differences between Porter’s illustration and the story line of The Toolbox Revisited : (1) the dependent variable 
(bachelor’s degree attainment) and most of the independent variables in this data essay are dichotomous, hence, our 
method is logistic, not linear, (2) Porter’s model assumes that students attend only one institution, whereas more 
than half the students in this analysis attended two or more, and (3) while multilevel analyses can create models for 
institutions "even for schools with few student observations" (Porter 2005, p. 110), of the 3,258 institutions in the 
NELS:88/2000 postsecondary transcript files, 1,003 had only one student in the sample. An institution-level model 
with a population of one is out-of-scope. 
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With the student as the unit of analysis, and 25 percent of the students in the universe we are 
examining attending more than two institutions as undergraduates, the calculation of institutional 
effects becomes problematic. For the students who attended only one four-year college, the 
weighting of institutional characteristics such as those Titus (2004) admits into his analysis of 
three-year persistence at the same institution (e.g., control, size, selectivity, degree of 
residentiality, and percent female, etc.) are fairly easy to plot. But for those who attended more 
than one institution, a weighting scheme for the influence of each institutional characteristic 
based on the proportion of undergraduate time each student spent in different institutions would 
be necessary. For a student who earned 26 credits at a community college, 30 credits at a four- 
year baccalaureate residential college, and 75 credits at an urban university, the ratios would 
dilute the very meaning — let alone effect — of any single institutional characteristic. This 
example is not a fantasy. Consider, for example, the credit accumulation at each school attended 
by two sets of students from the NELS:88/2000 who attended three institutions as undergraduates 
(fig- 2). 

Figure 2. Examples of credits earned at each institution attended by 1992 12th-graders 
who earned standard high school diplomas by December 1996, attended a 
four-year college at some time, earned a bachelor’s degree, and attended 
three institutions as undergraduates, by type of transfer 

Credits earned at each school attended 



Student cases 


School #1 




School #2 


School #3 


Community college transfers 


#1 


66 (community 


coll) 


34 (four-year) 


62 (four-year) 


#2 


32 (community 


coll) 


28 (community coll) 


90 (four-year) 


#3 


41 (community 


coll) 


18 (four-year) 


64 (four-year) 


Four-year-to-four-year transfers 2 


#1 


24 




36 


92 


#2 


36 




21 


88 


#3 


25 




30 


79 



a All three schools attended by these three students were four-year colleges. 

SOURCE: National Center for Education Statistics. NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 



How does one evaluate indirect effects of institutional characteristics and experiences (class size, 
contact with faculty, peer group interactions, etc.) for these students? Does one weight each 
potential effect by the percentage of credits earned at each school in relation to all credits earned? 
Can one somehow arrive at a consolidated index for each effect across schools? How would one 
compare these indices to those for students who attended only two institutions? To those for 
students who attended only one? All six of the students in figure 2 above earned bachelor’s 
degrees. For students who did not earn bachelor’s degrees but who attended three institutions, 
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what is the minimum threshold of credits one would set at each institution attended before trying 
to detennine the range of institutional characteristics and student experiences that might have 
some degree of association with the dependent variable? Across all patterns of inter-institutional 
"traffic" (Astin and Astin 1993), is there a common effects metric that encompasses mere 
excursions (incidental attendance at second or third institutions), true migration (formal transfer), 
and the nomadic behavior described as "swirling"? The author tried to develop a descriptive 
framework of excursion, migration, fragmentation, and discovery for this task, but with less than 
satisfactory results (Adehnan 2004b). Certainly, there are more sophisticated models that can 
address the rhetorical questions above, but they risk both neglect or devaluation of life-changing 
experiences at institutions where a student spent comparatively little time and earned few credits, 
and false identification of effects that can be very fleeting. 

Postsecondary student attendance patterns have rendered consolidated institutional-effects 
analyses moot. For that reason, these analyses are best carried out within the context of 
individual institutions (no matter where else the student goes to school). This study does not 
belittle the constructs of institutional effects and assessment efforts such as those of the National 
Survey of Student Engagement (NSSE) at all. For college and community college 
administrators, these assessments count a great deal in detennining the kinds of environmental 
adjustments likely to intensify student involvement with institutional services as well as to 
heighten student satisfaction with instruction, even if the student is present for only 24 credits. 

Even so, many of the variables of institutional effects analyses (and that includes high schools as 
well as colleges and community colleges) are sufficiently beyond the control of administrators so 
that the practical implications of whatever paths one delineates as productive are limited. Yes, 
high schools can change the structure of opportunity-to-learn, e.g., by not even offering 
arithmetic or watered-down pre-algebra and simultaneously conveying a clear message to 
students that they can meet the challenge of higher level math (Lee, Croninger, and Smith 1997). 
And the postsecondary level can follow suit, e.g., community colleges can refuse to teach 
arithmetic or basic algebra. Neither of these strategies is micromanagement, and, however some 
may wish otherwise, neither is likely to be implemented on a large scale. At the postsecondary 
level, the very conditions of control shift: An administrator cannot change the fact that the 
institution was not the student’s first choice and that he or she is determined to transfer from the 
moment of first registration; there is no drug to prescribe for a student with severe homesickness; 
one cannot — and should not — prevent the student from changing majors for the second time; no 
college authority has any influence on the romantic life and angst of 20 year-olds that may affect 
their involvement with academic pursuits (Okun, Taub, and Witter 1986); administrators can’t 
sand down every potentially hard edge of their schools. Furthermore, there is a natural capacity 
limit to truly meaningful contact with faculty outside the classroom. To use examples from the 
National Survey of Student Experience, how many faculty can deans deploy for student 
participation in faculty research projects or for students to work with faculty on committees and 
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student life programs? 47 The larger and/or less residential the institution, the less the opportunity 
for student-faculty interaction in other than casual or ritualistic activities (Hu and Kuh 2002). 

To repeat: The NELS:88/2000 does not reach the details of student development or institution- 
specific student experiences accounted for in other research lines. The elaborate connections 
between student background characteristics, social and psychological predispositions, initial 
perceptions and responses to a particular postsecondary environment, strength of goal 
commitments, etc. in relation to not only the fact but also the pace of student progress toward a 
credential — all of which have been explored and documented by the touchstone giants of 
postsecondary student outcomes research (Astin, Pascarella and Terenzini, Cabrera, Tinto) — lie 
beyond the scope of national longitudinal studies. That is not an excuse for the "economic" 
reading, broadly construed, of academic momentum. We all know that the many investment 
decisions made by students along their paths through postsecondary education do not occur in 
social and psychological vacuums. But the archival data on which this study draws isolate 
moments in which student choice intersects the structures of opportunity offered by institutions 
whose first order of business — and first order reason for existing in our society and economy — is 
the generation, preservation, and distribution of knowledge. This is a story about taking 
advantage of that mission; it is not a story about growing up, although that happens along the 
way. 



47 Kuh et al. (2001) reported that less than 20 percent of seniors in 2000 indicated experiencing these modes 
of nonstandard faculty contact. By 2004, that proportion had not changed, while 58 percent of seniors indicated 
they had talked often or very often with faculty outside of class about their grades (a ritualistic interaction that 
shouldn’t carry much weight in assessments of student/faculty contact). See www.nsse.iub.edu/~nsse/2004_annual_ 
report/html/responses _senior sfi.html. (Accessed 7/31/05). 
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Part V 

Closing the Gap 

One of the principal objectives of the No Child Left Behind legislation as it works its way 
through grades K-8 is to close the gaps in achievement between minority and majority students. 
When that objective is extended to high schools, as this essay underscores in Part III above, we 
have no adolescent left behind. And when extended to postsecondary schooling, as the logistic 
narrative has done, our objective is to leave no young adult behind, either. While race/ethnicity 
itself was not a significant variable in that narrative, the fact of an unhappy gap remains in 
degree completion — which means that there are residual echoes of experience by race/ethnicity 
that affect education outcomes. It is not the intention of this monograph to explore these echoes 
(there are far more sophisticated techniques and far more knowledgeable researchers than the 
author to engage in that task). But if we focus on the academic story line, we can at least draw 
some parameters of possibility for assisting minority students. 

Two issues attendant on this observation are now addressed: our confidence in degree 
completion rates, and what might contribute to closing the gap between majority and minority 
populations in degree completion. 

Bachelor’s completion rates: Some remarkable agreements 

The most prescient and eloquent statement of the problems attendant on our romance with 
graduation rates was offered a decade ago by Ronco (1996): 

Performance, even in relatively straightforward terms like graduation and retention, 
eludes our attempts to measure it. Students no longer march lockstep through four years 
of college to graduation. The educational model of the new millennium will likely be 
characterized more by lifelong learning, where students move among various kinds of 
higher education institutions, stopping in and out as their lifestyles and educational needs 
dictate. Enrollment patterns like these make indicators like four- or even six-year 
graduation rates for first-time, full-time freshmen largely irrelevant. Transfer students 
deprive both the sending and receiving institution of retention and graduation credit. 

(p. 1). 

But accepting the object of policy affections for a moment, the most enduring context for the 
analyses of The Toolbox Revisited involves what its predecessor called the Dow Jones Industrial 
Average accountability measure of U.S. higher education: the proportion of entering 
undergraduates who earn a bachelor’s degree. There are very few truly reliable national portraits 
of degree completion because there are very few data sources of convincing scope and 
magnitude. The task requires a longitudinal study with a tested sampling design and matching 
weights. The technology for a full census of completion does not yet exist. 
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Ewell (2004) has reminded us that a student-based indicator of completion leads to assessments 
of the benefits of higher education to the entire society and economy. An institution-based 
indicator does none of that, though there is no question that the institution is responsible for 
contributing to the student’s discovery of intellect and path to attainment. While the student is 
obviously more important, institutions are partners. 

There are three recent national accounts of bachelor’s degree completion for students who 
started out in postsecondary education together that can demonstrate how these two indicators, 
student and institutional, play out. 48 Each account comes from a different data set: 

1 . The NELS:88/2000 that is the subject of The Toolbox Revisited. To repeat its 
major characteristics: The temporal term is a maximum of 8.5 years (1992-2000) 
from the month of entrance, no matter when that happens and at what enrollment 
intensity (full-time or part-time). This is a grade cohort, with all students roughly 
the same age. 

2. A six-year longitudinal study, based on the Cooperative Institutional Research 
Project’s (CIRP) data, that followed 1994 entering college freshmen to 2000 
(Astin and Oseguera 2002). While dominated by traditional-age students, this 
cohort is not wholly homogenous in terms of age at entrance, but is confined to 
those who started out, full-time, at 262 four-year colleges, weighted to represent 
beginning full-time freshmen at all public and not-for-profit private four-year 
colleges (for-profit four-year colleges are not included). 

3. Another 6-year longitudinal study, 1995/6 through 2001, conducted by NCES as a 
spin-off from the massive Congressionally-mandated National Postsecondary 
Student Aid Study. We have used this data set, the Beginning Postsecondary 
Students longitudinal study, previously. Beginning students of all ages and 
enrollment intensities (full-time, part-time, and mixed) are included. 

Let us set these three data sets side-by-side, adjust the two NCES databases to match both each 
other and the CIRP database, and then assess how much they disagree with each other. We will 
confine both the NELS:88/2000 and the BPS 95/96-2001 to those who started out in four-year 
colleges, limit the dates of entry of the NELS:88/2000 cohort to the same year (1992) and, most 
importantly, put an upper bound on age at date of entry in the Beginning Postsecondary Students 
of 20. Table 30 offers these comparisons. 



48 There are other large longitudinal studies of degree completion, but their samples of institutions and 
students are not representative. For example, Saupe, Smith and Xin (1999) used the Consortium for Student 
Retention Data Exchange information from 174 public four-year colleges to track six-year (1989-96) institutional 
graduation rates. While these 174 institutions enrolled nearly half of entering four-year college students in 1989, 
they represent a class, not a universe. For another, but more dated example, Stoecker, Pascarella, and Wolfle 
(1988) used a 1980 CIRP follow-up to an initial 1971 sample of roughly 10,000 students who had attended only one 
four-year college in the interim. Some 487 college and universities were represented. Stocker, Pascarella, and 
Wolfle were interested in institutional effects, so their restriction of the universe to students who had attended only 
one school is justified, and their findings revealing. But that doesn’t justify using the conclusions as a generalized 
model of degree completion — certainly not under contemporary conditions of student mobility. For state-level 
postsecondary longitudinal studies, only Florida has produced data sets comparable to the national accounts. 
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Given the conditions under which degree completion is judged, the conclusions of these three 
data sets do not differ that much from each other. We can say that roughly a third of traditional- 
age students who start in a four-year college will earn a bachelor’s degree from the same school 
in the traditional four-year period, and that between 54 and 58 percent will earn the degree from 
the same school in which they began within six years of entry. When the option of earning a 
degree from a different four-year college than the one in which these students commenced study 
is included (in the NELS:88/2000 and BPS 95/96-2001), six-year degree completion rates for 
traditional-age students are in the 62-67 percent range. Only when the temporal boundaries are 
extended to 8.5 years in the NELS:88/2000 does degree attainment for those who started in four- 
year colleges approach 70 percent, and Florida state longitudinal studies show a similar rate 
(Johnson, Coles, and Thomas 2004). 

Table 30. Bachelor’s degree completion rates for students who began in four-year colleges 
according to three different national longitudinal studies of the 1990s 



Percent completing bachelor’s degree 



Bachelor’s degree 
comDletion modes 


NELS:88/2000 

1992-2000 


Cooperative 
Institutional 
Research 
Project (CIRP) 
1994-2000 3 


Beginning 

postsecondary 

students 

1995-2001 


Bachelor’s from same school 
in 4 years 


30.9(1.14) 


36.4 


33.1 (1.3) 


Bachelor’s from a different 
school in 4 years 


3.0(0.30) 


Not available 


2.3 (0.3) 


Bachelor’s from same school 
in 6 years 


52.9(1.27) 


57.6 


53.7(1.2) 


Bachelor’s from a different 
school in 6 years 


11.3 (0.79) 


Not available 


8.1 (0.4) 


Bachelor’s from same school 
in 8.5 years 


55.3 (1.24) 


60. 6 b 


Not available 


Bachelor’s from different 
school in 8.5 years 


14.1 (0.84) 


Not available 


Not available 


Total degree completion: 


69.3 (1.16) 


60. 6 b 


61.8 (1.2) 



a As reported in Astin and Oseguera (2002). Standard errors are not available. 

b In Astin and Oseguera, this cumulative figure includes students who were still enrolled at their institution of first 
attendance at the end of six years. 

NOTES: Standard errors for the NELS:88/2000 and BPS95/96-2001are in parentheses. 

SOURCES: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement) and Beginning Postsecondary Students Longitudinal Study, 1 995/96—200 1 , Data Analysis 
System (NCES 2003-173). Astin and Oseguera (2002). 
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Degree Completion: How High Can It Go? 



If roughly seven out of ten traditional-age students who start in a four-year college — and roughly 
two out of three who ever attend a four-year college — graduate within eight and one-half years, 
how likely is it that the rate will improve, by how much, and among what groups? When 
looking back at the factors consistently contributing to bachelor’s degree completion in table 29 
and asking which are most subject to change by external parties with little to modest, but 
creative, effort, a few stand out, and encourage speculation. All depend on student response. In 
the words of Tinto’s (1987) astute guidance: 

Though . . . institutions owe each admitted student an equal degree of attention, it 
does not follow that institutions should be held accountable for the equal education 
of all admitted students. To absolve . . . [students] of at least partial responsibility 
for their own education is to make a serious error, (p. 135) 

Let’s look at each of the more promising levers, and then explore what the configuration of 
change in behaviors and policy might suggest for upper boundaries of degree completion and the 
closing of the gap in completion among race/ethnicity groups. 

1) Less than 20 credits in the first calendar year of enrollment: Approximately nine 
percent of those who entered four-year colleges at any time earned between 15 and 19 
credits in their first calendar year, with African-Americans and Latinos overrepresented. 49 
Within this group, only 35 percent (s.e. = 3.05) earned bachelor’s degrees compared with 
77.7 percent (s.e. = 0.99) of those who earned 20 or more credits. Fifteen to nineteen 
credits is close enough to 20 to be optimistic: If we know more about who students are 
and where they come from, colleges and community colleges can target dual enrollment 
efforts to yield at least six additive credits for these students, thus pushing them across 
the 20-credit line. But we do know something about these students, and our optimism 
must be tempered by the facts that 15.3 percent (s.e. = 2.49) of them were assigned to 
remedial reading in their first postsecondary year, an overlapping 34.5 percent (s.e. = 
3.93) never reached Algebra 2 in high school, and only 27.9 percent (s.e. = 2.96) earned 
any college-level mathematics credits in that first year. For the residual group (about six 
percent of those who enrolled in a four-year colleges at any time), suburban high school 
students from the Pacific and Southwest Central census divisions 50 are overrepresented, 
and provide some geo-direction for dual-enrollment policy targeting. 



49 Some 7.4 percent (s.e. = 0.56) of white students earned between 15 and 19 credits in their first calendar 
year of attendance, compared with 11.1 (s.e. =1.86) of African-Americans and 18.5 percent (s.e. = 3.30) of Latinos. 

50 The Pacific census division includes California, Oregon, Washington, Alaska, and Hawaii; the Southwest 
Central encompasses Texas, Oklahoma, Arkansas, and Louisiana. 
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2) Ratio of non-penalty withdrawals and no-credit repeat courses to all courses 
attempted: For about 10 percent of the NELS:88/2000 cohort who attended a four-year 
college at any time, this ratio was 20 percent or higher. That is, this group of students 
withdrew from or repeated at least one out of every five courses in which they enrolled, 
behavior negatively associated with degree completion. Compared with white students, 
all minority groups are overrepresented. 51 What would happen if institutions limited the 
number of no-penalty withdrawals and no-credit repeats students were allowed to 
accumulate, converting these cases above a given threshold to penalty grades? This is a 
risky proposition and difficult to model because we do not always know why students 
withdraw from courses after the customary drop/add periods (Adams and Becker 1990). 
Students repeat courses to earn better or passing grades. In the case of remedial work, 
the repeat is required, and the list of courses with the highest ratios of withdrawal and 
repeat grades is dominated by remedial offerings (Adehnan 2004a, table 6.6, p. 84). In 
other cases, repeats are a luxury. Since no-penalty course withdrawals and no-credit 
repeats increase time-to-degree measurably for those who earn degrees (Adehnan 2004a, 
table 6.2b, p. 79; Knight 2004b; Noxel and Katunich 1998), and since these behaviors 
have a rippling effect by blocking other students from seats, often in high demand 
courses, it is in institutions’ self-interest to limit the practice. Intensified advisory care to 
student credit loads (Szafran 2001) and more precise placement criteria should help. 

A good example of the way this might work is provided by Eno, McLaughlin, 

Brozovsky, and Sheldon (1998), who predicted difficulty ratings for specific courses 
likely to be taken by entering freshmen in their institution on the basis of both overall 
high school GPA and, more importantly, grades in specific high school courses. The 
hypotheses for each student within each course were then set against the empirical record 
of students’ actual performance in the college course at issue. The infonnation assists 
students’ advisers with indications of which courses a student might find particularly 
difficult. The likelihood of no-penalty withdrawals or no-credit repeat grades is 
attenuated by delaying students’ engagement with those courses until they have gathered 
the requisite momentum elsewhere in the college curriculum. 

3) Use of summer terms: As table 3 1 suggests, earning more than four credits during 
summer tenns may have a considerable influence on the degree completion rates of 
African-American students in particular. That is, 78.2 percent (s.e. = 4.12) of those who 
started in four-year colleges and exceeded this threshold earned bachelor’s degrees, 
compared with 21.2 percent (s.e. = 4.59) of those who did not earn summer-term credits 
at all. The difference was also significant for white students, but not to the same degree. 
Parallel comparisons for Latinos and Asians were not statistically significant. 



51 Some 8.3 percent (s.e.= 0.68) of white students were in the high ratio category, versus 12.2 percent 
(s.e. = 3.04) of Asian students, 12.3 percent (s.e. = 1.96) of African-American students, and 19.6 percent 
(s.e. = 3.56) of Latinos. 
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Table 31. Of 1992 12th-graders who started their postsecondary careers in four-year 
colleges, percentage who earned bachelor’s degrees by December 2000, by 
number of credits earned in summer terms, by race/ethnicity 



Percentage earning bachelor’s degrees by December 2000 
Number of summer-term credits 



Race/ethnicitv 


None 


1-4 




More than 4 


All 


56.2 (1.99) 


68.1 


(2.78) 


79.7 (1.29) 


White 


59.8 (2.22) 


74.2 


(2.58) 


82.2 (1.19) 


African-American 


21.2 (4.59) 


42.5 


(10.3) 


78.2 (4.12) 


Latino 


48.6(7.14) 


28.3 


(7.15) 


56.4 (6.21) 


Asian 


66.8(10.3) 


70.0 


(13.0) 


77.9 (7.08) 



NOTES: Standard errors are in parentheses. Weighted N=1.14M. 

SOURCE: National Center for Education Statistics: NELS:88/2000 postsecondary transcript files (NCES 2003-402 

and Supplement). 

Summer-term enrollment that results in more than four additive credits works for just 
about everybody (we saw that in the multivariate analyses). While it surprisingly does 
not shorten time-to-degree for those who earn degrees (see Appendix K), summer credits 
are associated with higher graduation rates for everyone except Latino students. It is thus 
in an institution’s self-interest to encourage the use of summer-term work, either at the 
institution itself or at another school. Ways of approaching this goal shy of tuition 
discounting include moving some of the offerings of high demand courses from the 
traditional academic-year terms into the summer term, offering summative course work 
in the major in the summer term, and recruiting students for credit-bearing internships 
and cooperative education placements in the summer terms. Academic administrators in 
cooperation with strategic enrollment managers can be very creative. 

4) The high school curriculum component of Academic Resources: One of the more 
dramatic illustrations of the potential influence of high school perfonnance on bachelor’s 
degree attainment in the original Tool Box isolated four major race/ethnicity groups, 
confined each to those who entered a four-year college directly from on-time high school 
graduation, and compared two estimates: the bachelor’s degree attainment rate for 
everyone versus the bachelor’s degree attainment rate for those who were in the top 40 
percent on each of the three component measures of Academic Resources (curriculum, 
class rank/GPA, and senior year test score). Though undercut by the fact that, in 
multivariate analyses, race/ethnicity does not play much of a role in bachelor’s degree 
completion, the cross-tabulation of these estimates suggested that the criterion-referenced 
curriculum variable had the potential to boost African-American degree completion by 28 
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percent and Latino degree completion by 18 percent (though that datum was statistically 
shaky) and white students’ degree completion by 10 percent. For Asian students of the 
period, test scores made more of a difference than either curriculum or class rank/GPA. 

Class rank/GPA and senior year test score are not criterion referenced, and while 
performing better in high school in terms of grades and tests improved degree completion 
in the original Tool Box, it was by lesser magnitudes. It was observed that “in a happy 
paradox, everybody can be in the ‘highest 40 percent’ on the [criterion-referenced] 
curriculum measure,” whereas that is not true for test scores or class ra nk (Adehnan 
1999, p. 85). Content counts, particularly for minority students, and the influence of 
special preparation programs supports that conclusion (Ishitani and Snider 2004) 

Does this analysis still hold, particularly as we have already observed improvements in 
academic curriculum participation by the High School Class of 1992 compared with the 
High School Class of 1982 (see tables 4 and 5 above)? That depends on what else is in 
the mix of factors associated with degree completion, and unlike the original analysis of 
this feature in Answers in the Tool Box, the other factors are matters of postsecondary 
entry and postsecondary academic behavior. 

How might the four factors just highlighted, along with direct entry to higher education following 
high school graduation, cumulatively influence bachelor’s degree completion for the core 
universe of this study, the subjects of table 29? To remind the reader just who these students are: 

• they were 12th-graders in 1992; 

• they earned a standard high school diploma (not a GED) by December 1996 (as 
the NELS transcript records conclude in December 2000, an honest account of 
degree completion should allow the student at least four years to reach that mark); 

• they attended a four-year college at any time (20 percent started in community 
colleges, and those who started in community colleges earned an average of 52 
credits [s.e. = 1.58] from community colleges and 59.5 credits [s.e. = 1.92] from 
four-year colleges); and 

• their postsecondary records were complete, i.e., no transcripts were missing. 

Table 32 takes this group, and first sets down a base bachelor’s degree completion rate for the 
entire cohort by four major race-ethnicity groups. 52 The table then enters five key conditions 
influencing degree attainment that are subject to the initiatives and control of second parties, and 
marks the potential bachelor’s degree completion rate for students meeting the thresholds of 
those cumulative conditions. As each condition is added to the cumulation, the universe narrows 
for the NELS:88/2000 cohort, and as it narrows, hypothetical degree completion rates rise. One 
could choose a different sequence of conditions, but the bottom line would be the same. 



52 

There were too few American Indians in the base group to follow through a narrowing population as 
conditions for academic momentum were added. 
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Table 32. Hypothetical cumulative consequences of variables critical to bachelor’s degree 
completion for 1992 12th-graders who earned a standard high school 
diploma by December 1996, attended a four-year college at any time, and whose 

postsecondary records were complete, by race/ethnicitv 

Percentage earning bachelor’s degree 

Cumulative African- 



conditions 


White 


American 


Latino 


Asian 


M 


l)Baseline, no 
conditions 


67.6 (1.18) 


52.1 (4.26) 


45.4 (3.74) 


67.9 (4.71) 


64.6 (1.12) 


2) No delay of entry 


71.0(1.22) 


54.6 (4.49) 


50.5 (3.79) 


68.2 (4.89) 


67.9(1.15) 


3) No delay, top 
40 percent of high 
school curriculum, 
and highest high 
school mathematics 
above Algebra 2 


85.6(1.50) 


65.9(8.57) 


69.2 (6.33) 


91.5 (1.96) 


84.1 (1.40) 


4) No delay, top 
40 percent of high 
school curriculum, 
and more than four 
credits in summer 
terms 


90.6(1.31) 


84.6(5.95) 


69.2 (8.12) 


92.6 (2.27) 


89.1 (1.30) 


5) No delay, top 
40 percent of high 
school curriculum, 
more than four 
credits in summer 


92.6(1.23) 


88.2 (5.28) 


71.9(9.07) 


93.9(2.16) 


91.4(1.24) 



tenns, and 20 or 
more credits in 
first calendar year 
of attendance 

6) No delay, top 
40 percent of high 
school curriculum, 
more than four 
credits in summer 
terms, 20 or more 

credits in first 95.5 (0.98) 94.3 (4.62) 79.4(11.1) 95.3 (2.20) 94.6(1.07) 

calendar year, and 

less than 10 percent 

of grades were 

withdrawals or 

no-credit repeats 

NOTES: Standard errors are in parentheses. Weighted Ns for each cumulative step: (1) 1.45M; (2) 1.33M; 

(3) 712k; (4) 621k; (5) 310k; (6T273k. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
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These are statistical coincidences, not causes. The subjunctive is very strong in this presentation. 
The standard errors indicate that not all these changes would be statistically significant, 
particularly for minority students, but that phenomenon, too, is a consequence of a shrinking 
population at each step. No delay of entry is invoked as the first condition because it results in 
the least shrinkage, leaving a larger population with which to assess the other four factors. 

All five conditions of table 32 are criterion-referenced. That is, everyone can meet those 
conditions; everyone can cross those thresholds. The populations could be much larger. Does 
that mean that future degree completion rates will look like those in this table if everyone meets 
the criteria on all five counts? No: We know that not everybody will make it. We have already 
marked and acknowledged that some students will flu nk out or become status dropouts for other 
reasons. But the table suggests just where the improvements are likely to be dramatic — and for 
whom. Some examples, by race/ethnicity, would be instructive. 

For Latino students, high school academic curriculum attainment and entering college directly 
from high school provide the greatest leaps in degree completion, narrowing the gap with white 
students from 22.2 percent in the base rate to 16.4 percent. But little else narrows the gap much 
further. Postsecondary summer-term participation, for an obvious case, yields something of a 
zero, and getting across the 20-credit line in the first year of attendance appears to do little for 
Latino graduation rates (though if the sequence of conditions were different, it would do more). 
If we are looking carefully at this fastest growing population in the United States, we pour our 
efforts into high school preparation above everything else. A decade from now, we will be able 
to assess better whether postsecondary behaviors become more impressive contributors. 

For white and Asian students, moving into the top 40 percent of the high school academic 
curriculum intensity index and completing high school mathematics beyond Algebra 2 also is the 
strongest engine among these variables for boosting college graduation rates. Earning more than 
four summer-term credits adds something for white students, but nothing to speak of for Asian 
students. The other factors may contribute minor momentum, but once one reaches a 
hypothetical 90 percent completion rate, subsequent improvements are not meaningful. 

For African-American students, who start out at a higher bachelor’s degree completion rate 
than do Latinos, the high school academic curriculum factor does not close the degree 
completion gap by a statistically significant amount, but earning more than four credits in 
summer tenns offers a stunning boost, narrowing the completion gap vis-a-vis white students 
from 15.5 percent to 6 percent. The momentum provided by this high-octane persistence 
behavior continues through the first calendar year credits criterion and avoidance of no-penalty 
withdrawals and no-credit repeat grades until, at the bottom line of the hypothetical rates set 
forth in table 32, African-American degree completion rates would be no different from those of 
whites and Asians. 

What did African-American students in the NELS:88/2000 cohort universe who attended a four- 
year college at any time study during summer tenns? A transcript-based account can be very 
revealing — and in this case, positively so. Using 1 10 aggregate course categories, table 33 
displays the 15 categories accounting for the highest percentages of summer-term credits earned. 
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Table 33. 



Percentage of credits earned during undergraduate summer terms by 
African-American 1992 12th-graders in aggregate course categories 



Category 



Percent of total summer-term credits 



Business other than accounting and administrative science 5.5 

Chemistry 5.1 

Physical education activities/health information 4.9 

College-level mathematics 3 4.1 

College-level writing b 3.7 

General biology 3.3 

Psychology other than general psych 3.0 

Physics 2.8 

U.S. history surveys, American Civilization 2.8 

Micro/macroeconomics 2.6 

Accounting 2.5 

Calculus 2.5 

Foreign languages 2.4 

Liberal studies/general humanities/general social science 2.1 

Oral communication/public speaking 2.0 



Total for top 15 categories 46.7 

“includes college algebra, finite mathematics, precalculus, and statistics. 

b Includes standard English composition, technical writing, creative writing, and advanced essay. 

NOTE: The calculation is a "credit ratio" in which the unit of analysis is the course category. All student-weighted 
earned credits from all course categories are added, with the total a finite glass of 100 percent that becomes the 
denominator for subsequent calculations. No standard errors are produced. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402). 



This is a very respectable list for anyone. It advises advisers: Point out to African-American 
students that their peers have proved (a) summer-term credit production is a benefit, and (b) the 
student can concentrate in the summer terms on the kind of critically demanding and critical 
gateway courses that might not otherwise be in as intense focus during an academic year term. 
The learning goes deeper under single-subject concentration. Examples include organic 
chemistry, calculus, and experimental psychology. To repeat: There are values in summer-term 
credit production for everybody, but if summer-term credit production looks like it benefits a 
particular group more than others, that needs to be said. If we don’t know why the benefits seem 
more significant for African-American students, let us submit the issue to future research. 

Is the story for low-SES students the same? 

Reviewers of drafts of this document asked whether the same type of analysis could be 
performed by socioeconomic status, since we know that the gap in degree completion between 
the highest and lowest SES quintiles is far wider than any pair of race/ethnicity comparisons 
(Adehnan 2004a, table 3.1, p.34). The lowest SES quintile group is small: only 6 percent of the 
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1.43 million 1992 12th-graders who graduated from high school by December 1996 with a 
regular diploma and who attended a four-year college at some time. These students completed 
bachelor’s degrees at a rate of 35.9 percent, compared with 55.4 percent for students in the 
middle (3rd) SES quintile, and 79.7 percent for students in the highest SES quintile. Perhaps the 
best illustration of what might close the gap for them is to compare their hypothetical rates of 
completion with students from the middle and highest quintiles, and table 34 does so. We use 
the same levers as variables because they are subject to second -party action. But we have to stop 
after the step where summer-term credits are added in because (a) in extending the data beyond 
the boundaries of table 34, subsequent variables in the cumulative sequence (first-year credit 
threshold and withdrawal/repeat limits) do not add anything to the degree completion rates of 
low-SES students, and (b) the standard errors of the estimates have risen to the point at which 
comparisons between the lowest SES quintile group and the middle group are not statistically 
significant. The data of table 34, though, send clear messages: Despite obviously great 
limitations inherent in low socioeconomic status, the most promising engine of momentum for 

Table 34. Hypothetical cumulative consequences of variables critical to bachelor’s 
degree completion for 1992 12th-graders from the lowest, middle, and 
highest socioeconomic status quintiles who earned a standard high school 
diploma by December 1996, attended a four- year college at any time, and 
whose postsecondary records were complete 



Cumulative Percentage earning bachelor’s degree 

conditions 





Lowest 
SES quintile 


Middle (3rd) 
SES quintile 


Highsest 
SES quintile 


1) Baseline, no conditions 


35.9(3.74) 


55.4 (2.34) 


79.7(1.50) 


2) No delay of entry 


39.9(4.18) 


59.7 (2.47) 


81.3 (1.46) 


3) No delay, top 40 percent 
of high school curriculum, and 
highest high school mathematics 
above Algebra 2 


58.7(6.14) 


72.5 (3.71) 


89.0(1.21) 


4) No delay, top 40 percent of 
high school curriculum, highest 
high school math above Algebra 2, 
and more than 4 credits in summer 
terms 


59.6(9.73) 


82.2 (5.26) 


91.2 (1.25) 



NOTES: Standard errors are in parentheses. Weighted Ns for each cumulative step: (1) 1.43M; (2) 1.31M; 

(3) 546k; (4) 260k. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402). 
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these students is a strengthened high school curricular background, and achieving that objective 
requires greater attention to the provision of curriculum and the quality of learning environments 
in schools attended by these students, no matter where those schools are located. Even then, the 
degree completion gap between the lowest and highest SES quintile would be 32 percent. 

The bottom line on degree completion 

In reporting the proportion of college students who complete bachelor’s degrees, there seem to 
be as many answers as there are observers. A number of rules are recommended: 

1 . Use transcript-based data sources. They do not lie; they do not exclude. 

2. Follow the student, not the institution. Community college transfer students are 
followed to determine degree completion. Do it for everybody! And if one insists on 
institutional graduation rates, give the institution credit for students who transfer in. 
We don’t do that now. 

3. Do not restrict the universe to students who entered higher education only in the 
fall term and only full-time (otherwise one excludes half of an entering cohort). 

4. Do not mix your 18 year-old daughter and your 34 year-old brother-in-law in the same 
analytic of degree completion. They live on different planets of life’s responsibilities. 
In other words, institutions should consider ledgers indicating degree completion rates 
separately by age bracket at the point of entry, and national accounts should follow 
suit. 

5. The only people for whom bachelor’s degree completion rates should be calculated are 
those who actually enrolled at a bachelor’s degree-granting institution at some time. 

All these common sense rules, respectful of students and their families, were observed in The 
Toolbox Revisited — and in Answers in the Tool Box before it. 

The elaborate multivariate analyses of Parts III and IV of this essay provided a framework within 
which academic momentum toward bachelor’s degree completion (not toward something else 
such as first-year GPA or second-year retention) could be tracked and judged, but without 
race/ethnicity as a vector. When we extract from this framework the most tractable variables and 
examine their potential repercussions in terms of closing degree completion gaps by 
race/ethnicity, there is no question that the academic intensity of secondary school curriculum 
continues to serve as the engine of subsequent academic momentum, and that the use of summer 
tenns for substantive, highly focused postsecondary coursework is an effective booster to that 
engine. Both these variables are mixtures of the student’s use of time and the mastery of content. 

But what does “academic intensity” or “an academically ‘rigorous’ curriculum” really mean? 
What content, postsecondary as well as secondary, could be labeled as “challenging,” that is, the 
knowledge toward which students eagerly reach as opposed to unenthusiastically slog through? 
These questions are part of a large missing component in this study. Part VI of this essay will 
reflect on that component with the greater care it deserves. 
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Part VI 

The Missing Element of This Story 

There is (at least) one major missing element of the story line in both the original Tool Box and 
The Toolbox Revisited, and some important questions readers might have asked in the course of 
the logistic narrative that should be addressed. The missing element, a by-product of the 
limitations of the NCES grade-cohort longitudinal studies database, is content standards in high 
school curricula. Postsecondary curricula, which are far more complex in organization, offer a 
different framework for assessing potential student learning. 

Beyond Course Titles to Content Standards 

In the years following the release of A Nation at Risk (1983), a standing one-liner on the lecture 
and conference circuit ran something like “we’ve changed the marquee on the theater, but the 
show inside is still the same.” The marquee carried the course titles, and for better or worse, that 
is what our grade-cohort longitudinal studies transcript files rely on. As Shireman (2004) has 
concisely put it, "If schools just change the names of the courses . . .students will not have 
learned anything more" (p. 4). In building the NELS:88/2000 data files, we discovered that in 
some high schools, “precalculus” on a transcript could mean any mathematics prior to calculus, 
including Algebra 1 . On the postsecondary transcripts we often ran across cases of professorial 
marketing with course titles such as “Tooth Brush,” “Dots to Dinosaurs,” “Time After Time,” 
and (yes) “Good Books” that even online catalogues could not explain. The postsecondary 
transcripts also carried 3-credit courses in topics such as social event planning, daily living 
skills, and "appreciation of sports" that, when the syllabi were examined online, could be offered 
to junior high school students. One has to acknowledge the limitations of the data source, in 
addition to its virtues. But the limitations, in these cases, were serendipitous because the online 
search for concrete clues to the content of course work was dispiriting. 

Over the past two decades, states have paid increasing attention to content standards for 
elementary and junior high school curricula, and the No Child Left Behind legislation has 
stimulated a period of intense review and refinement of these standards. But as Achieve pointed 
out in its review of state high school graduation requirements (2004), similar detailed sets of 
content standards at the secondary school level are rare. We get generalized requirements that 
reference course titles such as "1 lth-grade English" or "Applied Biology," and have no idea what 
precise learning objectives will be pursued. There have been exceptions, of course, with 
mathematics being an often revisited subject and in ways that demonstrated the difference 
between what should be expected of all high school graduates and what should be expected of 
entering postsecondary students (California Education Roundtable 1997). 

The past five years in particular have witnessed the birth and expansion of a number of large- 
scale efforts to get beyond Carnegie unit credit counting to comprehensive criterion-referenced 
statements of what graduating high school students should know and be able to do in order to 
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"succeed" in postsecondary environments. By "succeed" is meant completing degrees (see, e.g., 
American Association of Universities 2003; Achieve 2004). We have also witnessed the growth 
of high profile alternative approaches to secondary-postsecondary transitions such as "early 
college high schools." These efforts have been aided and abetted by comparative analyses of 
emerging state high school exit examinations and the content and knowledge-objectives of lower 
division college courses (Conley 2003; Venezia, Kirst, and Antonio 2003). The American 
Diploma Project (Achieve 2004) pushes beyond general knowledge-objectives to include 
samples of college assignments (e.g., profit-maximizing output analyses in microeconomics, pH 
calculations for a complex solution in introductory chemistry, an essay assignment on Plato’s 
distinction between thinking and belief in introductory philosophy) and workplace tasks (e.g., a 
bank loan officer’s assessment of an application from a corporation for $1.7 million to purchase 
two corporate aircraft, a report requiring measurements of DC supply voltage for diffusion 
furnaces in semiconductor manufacturing and analysis of furnace regulator modification costs, 
and the determination of dosages in an insulin therapy regimen). These are superb examples of 
digging below the credit count to the stuff of learning. 

Granting that the academic quality and intensity of one’s high school curriculum is a key 
determinant of postsecondary success, there is no assurance that either the standards of 
secondary school performance, content coverage, or challenge of the material will come close to 
the threshold demands of either four-year or community colleges. For the vast majority of high 
school graduates, who will not attend selective institutions, the "disconnect" is considerable. 
Indeed, Venezia, Kirst and Antonio (2003) urge everyone to pay more attention than that to 
which the media are accustomed to the "broad access colleges," including community colleges, 
because the service of these institutions to the bulk of the nation’s postsecondary learners has the 
greatest potential to better "the civic and economic well-being" of every state and region (p. 46). 

It is with that in mind that the work of the American Diploma Project (2004) is noteworthy 
because the sample assignments and examination questions selected signal precisely the kind of 
learning expected of the bulk of the nation’s postsecondary newcomers. Such assignments and 
questions provide clear expectations for students entering community college occupational 
programs as well as those moving into the general education portions of postsecondary 
education. The microeconomics problems come from a community college, the chemistry from 
a research university. 

Previews of the future and their risks 

In fact, it could be argued that these previews of lower-division postsecondary learning 
objectives and tasks should be part and parcel of 1 1th- and 12th-grade curricula, equally 
accessible to students intending traditional lines to a bachelor’s degree and those following career 
and technical education paths that may include the bachelor’s, but certainly involve the 
occupationally oriented community college associate degrees. We can get to these levels through 
a modest amount of dual-enrollment for many students, but for those for whom dual enrollment 
is inaccessible, special postsecondary preview modules could be offered either by high school 
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faculty with the requisite knowledge or by visiting college and community college faculty. 53 
Somehow, students should have a taste of what will be expected, and whether their current 
knowledge and skills puts them at least on the threshold of those expectations, if not beyond. 
Some may argue that such previews will scare some students away from continuing their 
education or that some secondary school teachers will resist the intrusion. One has to be 
optimistic in both cases, and come to the task with the conviction that it’s worth the risk. The 
risk to academic momentum of not providing these opportunities is greater. 

While each state bears principal authority and responsibility for linking the curricular pathways 
of secondary and postsecondary sectors (Venezia et al. 2005), the examples set forth by the 
American Diploma Project and the “previews of the future” strategies described above do not 
treat state borders as Maginot lines. And for good reason: Roughly one out of four 
undergraduates (and one out of three African-American students) in The Toolbox Revisited 
universe started their postsecondary careers in a state other than that in which they graduated 
high school. The geographic mobility of our traditional-age postsecondary populations suggests 
that, at the least, multistate regions (e.g., South Atlantic) should be considered productive 
information zones for providing those concrete signals of expectations and specific examples of 
quality postsecondary student work. 

Moving the messages to students, families, and school teachers 

Let’s take the best of what is offered by reports such as that of the American Diploma Project 
and books such as Conley’s recent (2005) College Knowledge, and move it out to much larger 
audiences than policymakers and others who habitually read such reports and books. It might be 
more helpful for each college and community college to include in its information packages for 
prospective students a sample of those examination questions and assignments in courses 
typically taken by lower-division students. One might go even further and provide examples of 
exemplary student responses — even less-than-exemplary student responses — to those 
examination questions and assignments. 



53 

Dual enrollment has made huge strides since the NELS:88/2000 cohort went to high school. Some 38 
states now have formal policies (Karp, Bailey, Hughes, and Fermin 2004), though these vary widely in terms of 
what is offered and by whom, who pays tuition, who gets credit toward what, academic eligibility, grade levels, 
course enrollment limits, and whether credits earned are placed in escrow (though that varies more by institution 
than state). The 2002-03 NCES survey of dual enrollment practices at postsecondary institutions distinguished 
between high school students who took courses within formal agreements and those who took courses outside of 
those agreements. Our NELS:88/2000 transcript data cannot make that distinction, so they lump together all 
postsecondary credits earned at colleges or community colleges prior to the date of high school graduation. 

Kleiner and Lewis (2005) estimate that 813,000 high school students took courses at postsecondary institutions, 
either within and/or outside of dual enrollment programs and agreements, in 2002-03, compared with the weighted 
213,000 NELS:88/2000 students who earned any postsecondary credits through dual enrollment during their high 
school years (1989-92). While not exactly comparable populations, those figures are testimony to the growth of 
student participation in these arrangements. 
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A quarter century ago, the Educational Testing Service worked on developing an experimental 
examination to assess lower division college competencies across a matrix of cognitive 
operations (e.g., analysis, synthesizing) and general curricular areas (e.g., science, humanities). 
Called “Academic Competences in General Education,” the questions were open-ended (not 
multiple-choice) and required responses that could be composed in roughly 10 minutes. For 
example, one question went something like this: 

Suppose a new method for producing energy had been developed that, when brought 
on-stream, would have the effect of slowing the rotation of the Earth from 24 to 26 
hours. Before we can flip the switch on this new form of energy, an Environmental 
Impact Statement (EIS) must be filed. What would be the chapter headings and major 
sub-headings of that EIS ? You need not fill in the details, but the most important 
questions about the potential effects of a 26-hour day ought to be included. 

In a trial of the examination, faculty sorted student responses to questions such as “Earth’s 
Rotation” into piles of descending quality. They then described what the students whose 
responses were rated most highly did to earn that rating, turned to the second most highly rated 
collection of responses and arrived at a consensus of what these students did not do that the first 
group did — and so on down the line of the piles of student papers they had rated until a level 
euphemistically described as “nonresponsive” was reached. The faculty descriptions were 
subsequently aggregated and ironed out as perfonnance criteria. One could then turn to 
incoming freshmen with this question, along with an example of a first-rate response and an 
example of a fourth-rate response (out of seven levels), show them the performance criteria and 
say, “This is what you can expect to be able to do by the middle of next year; would you be 
better than the fourth level now?” A light goes on somewhere — or should — when entering 
students see that what it takes to respond effectively to the "Earth’s Rotation" question draws on 
a selection of knowledge from economics and business, psychology, international relations, 
world history, cultural anthropology, physical geography, technologies, visual arts — all in 
addition to the core sciences. More important, the light goes on when students see a good 
example of how to put all this together in an outline of a document that is as much a product of 
daily work life as it is a college exercise. 

And a similar light can go on among high school seniors as well, nearly all of whom think they 
are going on to something called “college,” but have little idea what that means beyond the fact 
that they will no longer be bound by school rules. Prospective students are not the only audience 
for this information: Their families and high school teachers are equal players. Not all parents 
will understand the assignments, laboratories, and examination questions, but certainly they will 
sense, from the examples, that postsecondary education is serious business, that their children 
will not get through with a carefree study schedule, that colleges and community colleges have 
standards, and that such examples are typical of what their children will learn. Inevitably (being 
optimistic about this) they will be proud enough at the prospect to ask their children to be 
prepared. High school teachers reading these assessment "prods" will have to reflect on the 
extent to which their graduating seniors possess the academic foundations and momentum to 
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respond successfully. To the extent to which high school teachers have doubts, they may adjust 
the form and content of their own assignments, laboratories, and examinations. Students are 
bound to be the beneficiaries of the resulting "postsecondary practice runs." 

Postsecondary dimensions of content 

At the postsecondary level, the issue of content is somewhat easier to address, but only in the 
context of a student’s entire undergraduate record. We know that students completing organic 
chemistry enrolled in the course not only with general chemistry as a background, but also 
mathematics at at least the precalculus level. We know that students completing a course in 
American economic history would have a very difficult time without prior learning in both U.S. 
history writ large and the standard introductory micro/macroeconomics sequence. A course in 
the psychological and cultural components of health care in a community college allied health 
program will probably not be offered to anyone without previous course work in general 
psychology. The chances that a fine arts degree candidate’s transcript shows an entry for a 
course in color and color theory without prior foundations and studio courses are remote. In 
higher education, we can use the second- and third-level courses in a field as rough 
confirmations of learning at the introductory level. There is a sufficient lattice-work within 
disciplines to track traces of content from one point on the scaffolding to another. No, the traces 
inherent in second- and third-level courses in a field are not in themselves content standards, and 
nuance is expected from school to school, but there is a generalizeable quality to these content 
cues. In fact, the generalizeable portions of upper-division curricula are what one is likely to see 
on the Graduate Record Examination Board’s subject matter examinations. This topic witnessed 
a surge of interest in the 1980s (for a paradigmatic analysis, see Oltman 1982), and is worth 
renewed exploration. 
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Part VII 

Concluding Messages 



Compared to its predecessor, Answers in the Tool Box, the preponderance of the Toolbox 
Revisited story has been on the postsecondary side of the matriculation line. Implicitly, it calls 
on colleges, universities, and community colleges to be a great deal more interventionary in the 
precollegiate world, to be more self-reflective about the paths they offer from high school 
through their own territories. It also calls on them both to fortify their institutional research 
capacities and integrate them more intimately with academic advising and course scheduling. As 
noted above, we are witnessing measurable ferment on the high school side of the passage, and 
as the principles of the No Child Left Behind legislation move beyond grade 8, we will see more. 
The higher education sector cannot sleep through these changes. 

To students as agents of their own futures 

Beyond that fundamental banner of institutional fortification, there are three sets of messages 
impelled by both studies. The first set is for entering high school students who, when asked, 
blithely shrug that "of course I’m going to finish college." 

1 . Just because you say you will continue your education after high school and earn 
a college credential doesn’t make it happen. Wishing doesn’t do it; preparation 
does! So . . . 

2. Take the challenging course work in high school, and don’t let anyone scare you 
away from it. Funny thing about it, but you learn what you study, so if you take 
up these challenges, your test scores will inevitably be better (if you are worried 
about that). If you cannot find the challenge in the school’s offerings, point out 
where it is available on-line, and see if you can get it that way. There are very 
respectable Web sites offering full courses in precalculus, introductory physics, 
humanities, music theory, and computer programming, for example. 

3. Read like crazy! Expand your language space! Language is power! You will 
have a lot less trouble in understanding math problems, biology textbooks, or 
historical documents you locate on the Web. Chances are you won’t be wasting 
precious credit hours on remedial courses in higher education. 

4. If you don’t see it now, you will see it in higher education: The world has gone 
quantitative: business (obviously), geography, criminal justice, history, allied 
health fields — a full range of disciplines and job tasks tells you why math 
requirements are not just some abstract school exercise. So come out of high 
school with more than Algebra 2, making sure to include math in your senior year 
course work, and when you enter higher education, put at least one college-level 
math course under your belt in the first year-no matter what your eventual major. 
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5. When you start to think seriously about postsecondary options, log on to college 
and community college Web sites and look not so much for what they tell you of 
how wonderful life is at Old Siwash, but what they show you of the kinds of 
assignments and examination questions given in major gateway courses you will 
probably take. If you do not see these indications of what to expect, push! Ask 
the schools for it! These assignments and questions are better than SAT or ACT 
preparation manuals in tenns of what you need to complete degrees. 

6. See if your nearest community college has a dual-enrollment agreement with your 
school system, allowing you to take significant general education or introductory 
occupational courses for credit while you are still in high school. Use a summer 
tenn or part of your senior year to take advantage, and aim to enter higher 
education with at least six credits earned this way — preferably more. 

7. You are ultimately responsible for success in education. You are the principal 
actor. The power is yours. Seize the day — or lose it! 

Given the story lines of The Toolbox Revisited, it is obvious that students are partners in their 
own education fate, and shouldn’t wait around for someone else to do something to them or for 
them. 

Public discourse, part 1: Dissonant data and their discontents 

The second set of messages is for those who engage in public discourse on education in general, 
secondary-to-postsecondary transitions, and ultimately, degree completion rates (with all stops 
in-between). We have some problems here. 

Foremost among these problems is the sheer volume of dissonant statistics that are thrown 
around about student progress, and all the labels of “at risk,” “minimal college-qualified,” and 
"failure" that get pasted to populations in the process. The “at risk” labeling default has gone so 
far as to turn students into “patients,” whose “illnesses” must be diagnosed and followed up with 
early intervention, intensive intervention, and continuous intervention (Seidman 2005, p. 298) 
that may even continue after graduation — and for “a modest fee” (p. 299). The data dissonance 
and deficit language cloud perceptions and preclude constructive policy. We all have 
considerable cleaning up to do. 

On any given day, the public will be offered a half-dozen different statistics on high school 
graduation rates, college-enrollment rates, college completion rates, grades, and time-to-degree. 
The data will appear in respectable academic journals in articles that were reviewed by peers 
who often are experts on statistical technique and (at best) novices on the data sources. Or they 
will appear in publications and on Web sites of respectable organizations, even though they were 
never reviewed by anyone outside the organization. Anything that appears between respectable 
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covers is taken as authoritative, and once it moves into the mainstream press and onto the home 
pages, we read the headlines but not the footnotes. Inference runs rampant. 54 

For any of these statistics, we never ask who is in the denominator: that is, who are we counting, 
and who are we not counting — and how? As a consequence, what often pours out are scare 
stories that make for good press and bad policy. The bad data-driven scare story, in fact, has 
become the preferred narrative. We are scared by stagnant high school graduation rates over a 
30 year period during which the size of the grade cohorts declined significantly then expanded 
dramatically with the baby boom echo, and during which we witnessed increased immigration 
from countries with mandatory school attendance ages much lower than ours. By an alternative 
view, it’s amazing we have maintained a stable high school graduation rate (the quality of high 
school curriculum aside). The same alternative view could be advanced with reference to rates 
of postsecondary credentialing: It’s remarkable we are maintaining the same degree-granting 
rates in the face of significantly higher enrollments (unless, of course, we are awarding an excess 
of cheap degrees). 

Dissonance by age and season: A plea for honest tracking 

The source of many unnerving postsecondary stories is one of the most grievous errors in 
analyses of student progress: including in the denominator students who started their 
postsecondary careers at age 29, 36, or 47 along with the mass of students who entered the 
postsecondary universe at age 18 or 19. Common sense says that a 19-year-old and a 31 -year- 
old are on completely different life trajectories, and the national data from the Beginning 
Postsecondary Students longitudinal studies back up the common sense. When the newspaper 
story uses the term, "college students," most adults think of their children, not their brother-in- 
law or their coworker. Community and four-year college administrators know the difference, 
and provide academic programs, scheduling and services for those different populations. 

But what are they to do when the press and the news Web sites complain that nearly half of 
entering students do not return for their second year or that the graduation rate is only 50 percent 
(thus assuming everyone else is a dropout), and they are called before legislative committees and 
boards of trustees to explain? There is an enormous difference by age at entry to the 
postsecondary system in these measures, and an even greater distortion when one restricts the 



54 For example, consider the following statement in a respectable publication: "One of the key reasons that 
low income students have such low completion rates in postsecondary education is that many work long hours in 
order to be able to afford college. They struggle to balance work with part-time enrollment in college ..." (Allen, 
Goldberger, and Steinberg 2004, p. 222). The data source for this assertion is the National Postsecondary Student 
Aid Study of 2000, a one-year snapshot that includes no "completion rate" data. Analyses of the NPSAS 2000 data 
files show that the statement does not reach the threshold of justification unless one divides the population by age 
bracket. At that point, one finds that, and among traditional-age students (presumably the group referenced by the 
scare), poor kids are no more likely to be working longer hours at their jobs than anyone else, though they are more 
likely to use their wages for education expenses. That, at least, is an honest statement — for a snapshot population. 
And it is not what we really would want to know. 
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definitions of what it means to "return to" or "graduate from" to those who started in the fall 
tenn, full-time, and who came back to or earned a degree from the same school. That 
denominator knocks out half of traditional-age students from the calculation, and denies the 
realities of geographic mobility that the Bureau of the Census — let alone NCES longitudinal 
studies — has documented for the 20-something population (Schachter 2004; Adehnan 2005b). 
Policies designed to "retain" students who have already moved to another state or who are 
cle facto ghosts by not being included in the retention denominator in the first place are, at best, 
wastes of energy. 

What is not a waste of energy is the task of developing more universal and efficient student 
tracking systems, and recapturing the headlines from the mongers of scare. There are those who 
will not accept NCES national longitudinal studies on the grounds that they are samples (no 
matter how scientific the sampling design), that we can only afford to start one every six or 10 
years, and then have to wait for people to age and accumulate academic history by which time, 
the grievance goes, "the data are old." Impatient to simulate instant longitudinal cohorts, they 
impute sequences of data from different sources and with denominators that include 
"projections," and produce shock data that cannot be validated by any sensible reference points, 
e.g., that only 18 percent of ninth-graders will earn an associate or bachelor’s degree within the 
subsequent ten years (National Center for Public Policy and Higher Education 2004). 55 

But even the best of state tracking systems and the services of the independent National Student 
Clearinghouse information system that currently (2005) covers about 2900 institutions (and cites 
a burgeoning interest in including high schools in the universe), will not produce the wealth of 
information that a NELS:88/2000 or a Beginning Postsecondary Students study yield. This 
essay cannot recommend policy in these matters, but it can recommend creativity and 
cooperation, serious reading of the papers and reports from Florida’s tracking system (e.g., 
Whitfield and Howat 1999; Goodman, Latham, Copa, and Wright 2001; Goodman, Copa, and 
Wright 2004; Johnson, Coles, and Thomas 2004), and reflection followed by activistic 
innovation, and will wager that the long-term results look better than the scare stories assume. 

Public discourse, part 2: The language we use 

Language does more than reflect reality — it creates reality as well. There are considerable 
problems with the language used in describing what happens to students in our education system, 
and our choice of terms sets boundaries and colors of reality. The boundaries and colors, in turn, 
condition the terms of policy. Let us illustrate with a few paired terms. These are contrary 
rhetorics, and this study frankly admits to taking sides in their contention. But it does so in order 
to urge a positive tone that, not so by-the-way, legislators, superintendents of schools, college 
presidents and other leaders would prefer to use. The language of leadership is a “can do” 
language, not a punitive rhetoric. 



55 There has never been a national longitudinal study of ninth-graders. But we do have a national 
longitudinal study of eighth-graders — the NELS:88/2000 — with transcripts, not imputations, projections, and 
dubious math. If we follow these eighth-graders, including high school dropouts, all the way through to age 26, 
ultimately 34 percent earned either an associate or bachelor’s degree (see the full account offered in Appendix L, 
table L12). That percentage at least puts us in range of doing better. If we accept the putative (and utterly false) 18 
percent, we risk abandoning all hope and effort. 
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“Attrition” versus “Persistence.” When “attrition” is the governing term, we worry about 
students who (it appears) leave school or college, and seek explanations for departure that have 
included theories of organizational turnover (Bean 1983) and failures of academic and social 
integration (Tinto 1987). At the first sign of exit — even though the student may return — we turn 
to negativity. There has to be something wrong here, we say. The student was “at risk,” the 
institution did not respond — we witness a cycle of blame. 

When “persistence” is the governing term, we take our directions from students. What did they 
do that resulted in attainment? What structures of opportunity do we need to offer so that future 
students can follow the same paths? What do we think works? Can we test it out? This is a far 
more positive approach. This essay endorses it: Drop "attrition," embrace "persistence"! 

“Retention” versus “Persistence.” Institutions “retain”; students “persist.” If our language is 
governed by “retention” all we see are institutions detennined to hold on to students, keeping 
them in places that may be unproductive, at all costs, and for the sake of their public ratings. If 
our language follows student “persistence,” on the other hand, we see those individuals making a 
series of rational choices that take advantage of the opportunities offered by institutions so as 
both to discover true interests and reach productive ends. Tinto would not object if the rhetoric 
of leaving an institution was turned into a saga of discovery. Students may go elsewhere; they 
may take extended time off from higher education; but ultimately they may judge the change as 
positive and not a result of failure (Tinto 1987, pp. 132-33). In the rhetoric of “retention,” 
students are passive: Something is done to them, and that "something" assumes a deficit model. 
Under the rhetoric of “persistence” they are actors shaping their fate, with a model of success in 
mind. Wouldn’t anyone rather have success? 

“Pipelines” versus “Paths.” As Bach et al. (2000) noted — and others have followed — there is 
no linear path to a degree, particularly for students who start out in community colleges. The 
default “pipeline” metaphor, used to describe presumably linear learning experiences and 
environmental sequences, is wholly inadequate to describe student behavior. Pipelines are 
unidirectional closed spaces, and under the “pipeline” metaphor students are passive creatures 
(as in “retention”) swept along or dropping out of the space completely through leaks at the 
joints. But student behavior doesn’t look like that at all: It moves in starts and stops, sideways, 
down one path to another and perhaps circling back. Liquids move in pipes; people don’t. 

At the high school level, for example, a student can acquire momentum in science through a 
combination of statistics and biology, on the one hand, or physics and calculus, on the other. 
These are different paths, but who is to say that, once in a four-year or community college, these 
students could not move in very different directions? The students entering a community college 
with the statistics and biology background thinking they were heading for further study in allied 
health fields could easily discover business and computer programming, and transfer to a four- 
year college to pursue an academic program in management information systems with both 
quantitative background and empirical habits of mind born of study in the life sciences. The 
paths to degrees offer many such intersections. 
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Under the “pipeline” metaphor, we look for easy (sometimes glib) causalities along a single line 
of explanation. “Paths,” on the other hand, allow for multiple analyses and discoveries of tools 
that suggest (but do not predict) productive routes to education goals. This essay obviously 
endorses "paths." 



Reiterations 

Virtually all reviewers of drafts of this study recommended a concluding reiteration of its major 
themes and conclusions. Three configurations of themes and conclusions stand out in response: 

First, there was a story about curriculum, the content of schooling, that was compelling in its 
secondary school dimensions in the original Tool Box, and is even more compelling now on both 
secondary and postsecondary stages. What you study, how much of it, how deeply, and how 
intensely has a great deal to do with degree completion. All of this is common sense, but 
requires equitable execution with emphasis on primary tools, which in this story means that: 

• Secondary schools must provide maximum opportunity-to-leam, by which we 
mean not merely course titles, but course substance. If we seek better preparation 
for any kind of postsecondary education — occupational, professional or 
traditional arts and sciences — we have to ratchet up the challenge of content. 

• Postsecondary institutions have got to be active players and reinforcers at the 
secondary school level — particularly in partnership with schools that are not 
providing or inspiring students — with opportunity to leam at those ratcheted-up 
levels of content. Pep talks, family visits, recruitment tours, and guidance in 
filling out application and financial aid forms are not enough. 

• Indeed, the first year of postsecondary education has to begin in high school, if 
not by AP then by the growing dual enrollment movement or other, more 
structured current efforts (for examples, see Hughes, Karp, Fermin and Bailey 
2005). If all traditional-age students entered college or community college with a 
minimum of 6 credits of "real stuff," not fluff, their adaptation in the critical first 
year will not be short-circuited by either poor placement or credit overload. 

Second, this curriculum story, joined by nuances of attendance patterns that turn out to have 
significant leverage, continues into higher education. These features of the saga of degree 
completion are rarely attended to, and all provide tools to enhance completion rates. 

• It’s not merely getting beyond Algebra 2 in high school any more: The world 
demands advanced quantitative literacy, and no matter what a student’s 
postsecondary field of study — from occupationally-oriented programs through 
traditional liberal arts — more than a ceremonial visit to college-level 
mathematics is called for. 
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• Academic advisers and counselors have to target every first-time student for at 
least 20 additive credits by the end of the first calendar year of enrollment. We 
saw the same consequences in the original Tool Box, though now we understand 
better that the chances of making up for anything less than 20 credits diminish 
rapidly in the second year. Community colleges have some special challenges 
here, given increasing rates of transfer among traditional-age students. With 6 
credits of dual-enrollment course work, even part-time students can reach 20 
credits in the first calendar year, and community colleges enroll the bulk of 
traditional-age part-time students. 

• Excessive no-penalty withdrawals and no-credit repeats appear to do irreparable 
damage to the chances of completing degrees. This phenomenon was also 
observed in the original Tool Box. Twice advised, institutions might think very 
seriously about tightening up, with bonuses of increased access and lower time- 
to-degree. 

• More than incidental use of summer terms has proven to be a degree-completion 
lever with convincing fulcrum. It’s part of the calendar-year frame in which 
students are increasingly participating. Four-year and community colleges can 
entice students into fuller use of summer terms with creative scheduling. 

Third, in contrast to their treatment in the mass of literature on academic progress, students are 
explicit, rather than implicit, in The Toolbox Revisited. They are respected adults playing large 
roles in their own destinies. What we call “variables” are not bloodless abstractions: they are 
signs of what students do; and our messages are about where and when the green lights and 
caution lights will flash along the paths toward degrees. While we trust that school and college 
actions will not leave them behind, they have equal responsibilities. 

Legacy 

These are limited beginnings of change in the terms of the enterprise with which any reader of 
this document is concerned. They are honest terms and do not pretend to predict, rather help us 
draw a background tapestry against which we can judge just how well we are doing for our 
children as they cross the cusp of adulthood. The terms derive from the story; the story derives 
from the wisdom of the U.S. Department of Education in establishing and maintaining its 
longitudinal studies; and our subsequent discussions and enlightenment derive from the 
leadership of the National Center for Education Statistics in executing those studies and 
providing us with archives of information that are the envy of other nations. All of this 
constitutes an unmatchable legacy. 
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APPENDIX A 



Changes in Demography, Geo-Demography, and Postsecondary 
Entry From the High School & Beyond/Sophomores to the NELS: 88/2000 

Table Al. Contrasts in the percentage distributions of selected demographic 

backgrounds, postsecondary plans, and postsecondary entry behavior 

of 1982 and 1992 12th-graders 

1982 (HS&B/So) 1992 (NELS: 88/2000) 



Parents’ highest level 
of education 



Less than high school graduate 


12.2 (0.47) 


8.1 (0.59) 


High school graduate 


32.4 (0.66) 


19.5 (0.79) 


Some postsecondary 


29.8 (0.58) 


41.2 (0.92) 


Bachelor’s degree 


12.7(0.45) 


17.0(0.71) 


Graduate or first professional 


12.9(0.55) 


14.2 (0.74) 


Race/ethnicity 


White 


77.5 (0.77) 


72.2 (1.32) 


African-American 


12.6 (0.63) 


12.0 (0.96) 


Latino 


7.1 (0.33) 


10.1 (0.89) 


Asian 


1.5 (0.16) 


4.3 (0.37) 


American Indian 


1.3 (0.19) 


1.5 (0.43) 


Proportion of same race 
in childhood neighborhood 


10 percent or less 


4.5 (0.26) 


6.3 (0.43) 


11-50 percent 


8.4 (0.43) 


11.7(0.63) 


51-85 percent 


16.8(0.59) 


24.1 (0.82) 


More than 85 percent 


70.3 (0.83) 


57.9(1.13) 


Proportion from second 
language backgrounds 


All 12th-graders 


5.5 (0.30) 


9.4 (0.84) 


White 


1.8 (0.19) 


2.0 (0.29) 


African-American 


1.2 (0.29) 


3.1 (1.41) 


Latino 


40.5 (2.18) 


50.9(3.12) 


Asian 


55.1 (4.11) 


48.0(3.48) 


American Indian 


17.9 (5.12) 


28.5 (16.7) 



See notes at end of table. 
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Table Al. 



Contrasts in the percentage distributions of selected demographic 
backgrounds, postsecondary plans, and postsecondary entry behavior 
of 1982 and 1992 12th-graders — continued 



1982 (HS& B/So) 1992 (NELS: 88/2000) 



Became parent by age 20 

Urbanicity of high school community 

Urban 

Suburban 

Rural 

Census division of high school 

New England 
Mid-Atlantic 
East North Central 
West North Central 
South Atlantic 
East South Central 
West South Central 
Mountain 
Pacific 

Highest level of education planned 

High school or less 
Postsecondary vocational 
2+ years of college 
Bachelor’s degree 
Graduate degree 

Entered postsecondary education 
within 8.5 years 3 of scheduled 
high school graduation 

Of those who continued to 
postsecondary, entered within 
7 months of high school graduation 13 



9.0 (0.41) 



20.2 (1.26) 
49.3 (1.55) 
30.6(1.31) 



6.7 (0.61) 
16.7(0.57) 
20.6 (0.60) 
7.9(0.41) 
16.1 (0.62) 
5.3 (0.20) 
10.1 (0.51) 
4.8(0.31) 
11.8 (0.55) 



19.5 (0.60) 
23.9(0.58) 
15.7(0.46) 
22.6(0.53) 
18.3 (0.53) 



59.0 (0.67) 



77.1 (0.74) 



10.8 (0.65) 



28.3 (1.53) 
40.9 (1.72) 
30.8 (1.66) 



4.6 (0.80) 
14.6(1.01) 

17.4 (0.84) 

8.5 (0.56) 
17.1 (0.90) 

6.6 (0.52) 

11.5 (0.70) 
6.3 (0.59) 

13.3 (0.78) 



5.9(0.52) 
12.0 (0.76) 
14.2 (0.64) 

35.5 (0.81) 

32.6 (0.90) 



77.3 (0.87) 



82.6 (0.82) 



a 8.5 years is the maximum time between the modal high school graduation date and the concluding date for the 
longitudinal study for the NELS:88/2000, that is, June 1992 through December 2000. The HSB/So is cut to match. 
b See definition of "no delay" in the glossary, p. 186. 

NOTES: Where applicable, columns may not add to 100.0 percent due to rounding. Standard errors are in 
parentheses. Weighted Ns: High School & Beyond/Sophomores = 3.3M; NELS:88/2000 = 2.6M. 

SOURCES: National Center for Education Statistics: High School & Beyond/Sophomore cohort (NCES 2000-194) 
and NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402). 
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APPENDIX B 



Principal Features of the NCES Grade-cohort Longitudinal Studies 

There are four grade-cohort longitudinal studies designed and conducted by the National Center 
for Education Statistics. Three of these have been completed: 

• National Longitudinal Study of the High School Class of 1972 (NLS-72), 

started with a cohort of seniors in the spring of 1972, concluded in 1986; 

• High School and Beyond, with a cohort of seniors in 1980, concluded in 1986, 

and another cohort of sophomores in 1980 (HS&B), concluded in 1993; 
and 

• National Education Longitudinal Study of 1988, initiated with an eighth-grade 

class in 1988 (NELS:88), concluded in 2000. 

The data from these studies are available in both public release and restricted (license required) 
form on CD-ROM, with electronic code books (ECBs) listing all variables, with descriptions and 
distributions. 

The fourth, the Education Longitudinal Study of 2002 (ELS:2002), starting with a sample of 
20,000 lOth-grade students in the spring of 2002, is in progress. 

Curtin, Ingels, Wu, and Heuer (2002) offer a figure with a temporal presentation of the four 
longitudinal studies, 56 highlighting their component and comparison points. Each of the studies 
begins with a national probability sample involving a stratified sample of schools and a random 
sample of students within the target grade in those schools. Schools in minority communities are 
over-sampled. In some cases, the samples are refreshed at later points in the longitudinal study 
(NELS:88 in 1990 and 1992) 57 and, in some cases, augmented at a later point (NLS-72 in 1973). 

Each of these longitudinal studies includes a great deal more infonnation than what is used in 
The Toolbox Revisited. Not all of them are comparable in terms of the depth with which various 
topics are explored. The surveys of the NLS-72 were focused wholly on students, whereas those 



56 Curtin, T.R., Ingels, S.J., Wu, S., and Heuer, R (2002). National Education Longitudinal Study of 1988: 
Base-Year to Fourth Follow-up Data File User’s Manual (NCES 2003-323). Washington, DC: U.S. Department of 
Education, National Center for Education Statistics (http://nces.ed.gov/pubs2002/2002323.pdf, p.3). 

57 

As Lucas and Berends (2002) remind us, the high schools attended by students in the NELS study are not 
a true representative sample. The reason stems from the decision to begin the longitudinal study in the eighth grade. 
When students move into high schools from the eighth grade (or, in some districts, from the ninth grade), it is 
impossible for the sampling framework to be based on high schools. Roughly 1,000 middle schools and junior high 
schools became 4,000 high schools by grade 10, and there was no way to follow the students into all 4,000 of those 
schools. So the NELS sample was refreshed in 10th and 12th grades to reflect the total enrolled population in those 
years (Ingels et al. 1994), and , by necessity, the refreshing was based in the high schools in which the vast majority 
of the existing NELS cohort students were enrolled. 
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of the subsequent longitudinal studies included parents, teachers, and secondary school 
administrators. The cognitive tests administered in the 12th grade to the NLS-72 cohort were 
administered in the 10th and 12th grades to subsequent cohorts, thus enabling measures of 
intellectual growth. High school course-taking for the NLS-72 was summarized and reported by 
the school, whereas for the HS&B/Sophomore cohort and NELS:88/2000 high school course- 
taking was derived directly from transcripts. And the postsecondary transcripts for the 
NELS:88/2000 were used to fill in missing infonnation from the high school transcripts in that 
cohort. Labor market histories were far more detailed in the NLS-72 and HS&B/Sophomore 
cohort than they were for the NELS:88/2000. Military records exist for the NLS-72 but not for 
any subsequent study. Student financial aid information included an unobtrusive Pell Grant file 
for the HS&B/Sophomore cohort, and that for the NELS:88/2000 included data from the 
National Student Loan Data System (though this file requires more cleaning and reconstruction 
in order to be truly helpful). 58 

Lastly, the shift from paper-and-pencil survey response forms to computer-assisted telephone 
interviews (CATI) in the 1990s constricted the range of questions asked (e.g., there was no time 
to ask students about reasons for changing majors, reasons for transferring from one college to 
another, and degrees of satisfaction with different aspects of postsecondary experience), whereas 
the NLS-72 paper survey forms covered these topics in some depth. 

Nonetheless, the archives of these data sets are the richest we have to explore the nature of 
secondary and postsecondary education and its consequences in the early adult life histories of 
Americans over the past 30 years. 



58 

See the brief discussion of financial aid data in the NELS:88/2000 in Adelman, C. Principal Indicators of 
Student Academic Histories in Postsecondary Education, 1972-2000. Washington, DC: U.S. Department of 
Education, 2004, p. 98. 
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APPENDIX C 



Differences Between the High School & Beyond/Sophomore Cohort and 
NELS: 88/2000 High School Record Variables, and Limited Imputation 

Procedures for the NELS: 88/2000 

The principal portions of the data sets in question are secondary school transcripts and allied 
information derived from survey questionnaires and computer assisted telephone interviews 
(CATI). The secondary school transcripts were collected and coded by the same contractor for 
both data sets, but the returns were very different. The High School & Beyond/Sophomore 
cohort began with a stratified national sample of secondary schools, all of which agreed to 
participate in the project and provide transcripts, access to teachers and administrators. The 
NELS:88/2000 began with a national stratified sample of schools with eighth grades, all of 
which agreed to participate in the project and provide appropriate information. As noted above 
in Appendix B, when students moved into high schools, the NELS sample was refreshed in 10th 
and 12th grades to reflect the total enrolled population in those years (Ingels et al. 1998), and, by 
necessity, the refreshing was based in the high schools in which the vast majority of the existing 
NELS cohort students were enrolled. The analysis file created for this study includes the 12th 
grade refreshed sample, but some of the high schools that were not part of the original stratified 
sample did not provide transcripts. 

At the postsecondary level, on the other hand, the transcript return rate increased from 89 percent 
for the HS&B/So to 93 percent for the NELS:88/2000 (Curtin, Ingels, Wu, and Heuer 2002), and 
the proportion of postsecondary participants with complete records increased from 88 to 97 
percent. 

Editorial processes 

The postsecondary transcript-based versions of both data sets were edited by the same person 
(the author of this study), and with the same rules, a feature that increases reliability at the price 
of intra-coder bias. If one thinks of two sets of archival documents: secondary and 
postsecondary transcripts for two cohorts, i.e., four data files, in only one did the editor actually 
see the paper documents submitted by institutions, namely, the postsecondary transcript files of 
the NELS. These artifacts contained infonnation that allowed corrections and fill-ins for missing 
information on the secondary school transcript files, e.g., state location of high school, high 
school graduation date, high school diploma type, Advanced Placement course credits (by 
examination), and SAT and ACT test scores. Hence, the presentation of secondary school 
transcripts on the restricted file with the postsecondary transcript data (NCES CD#2003-402) is 
more accurate than it is on earlier NELS restricted files. After a year of use, a supplement to 
NCES 2003-402 was issued in June 2004, containing further refinement of both high school and 
college transcript-derived variables, in addition to new derived variables. 
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Missing data and limited imputation 

The analysis files for both this study and Answers in the Tool Box used high school transcripts 
with complete records for at least grades 10-12. But the NELS:88/2000 transcript data, as well 
as test score data, fell short of the HS&Beyond/So coverage, even after preparation of the 2004 
Supplement to the original NELS Postsecondary Files that included the second round of 
corrections and fill-ins of missing infonnation on the high school transcripts. 



Table Cl. Unweighted percentage of 1982 and 1992 12th-graders missing precollegiate 
academic performance data 





HS&B/So 


NELS:88/2000 


After 

Supplement 


Performance variables 








Curriculum measure 


18% 


14% 


14% 


Class rank/GPA 


13 


20 


17 


Senior test score 


9 


14 


14 


One or more measures 


18 


24 


21 



SOURCES: High School & Beyond/Sophomore cohort (NCES #2000-194); NELS:88/2000 Postsecondary 
Transcript Files (NCES 2003-402 and Supplement). 



Class rank/GPA, the combined high school class rank/grade point average quintile, is the major 
problem in the comparison displayed in table Cl. In both data sets this variable (called 
"CLSSRNKQ" for programming purposes) was constructed in the same way, but the input data 
initially yielded a far smaller universe for the NELS than was the case for the HS&B/So. 

The construction of this variable begins with class rank, expressed as a percentile, for students 
whose high school graduating classes are greater than 10. Class rank was chosen for the base 
reference because it overrides variability in local grading practices. Not all high schools 
compute class rank, so for a significant percentage of students, this datum is missing. For these 
missing cases (as well as for the students from very small high schools), the variable 
construction turns to high school grade point average (where available, and only for students 
with three years or more of course work in all high schools attended). In both data sets, all 
known cases of high school GPA were weighted and set out in quintiles. The missing cases of 
class rank, by quintile, were then substituted by the GPA quintile (the correlation between the 
two quintile scales was .84). 

However, there are two significant differences in data input between the HS&B/So and the 
NELS:88/2000. First, the HS&B/So grade point average was for academic courses only, 
whereas in the NELS, the GPA applied to all courses. Second, by the time of the NELS 
histories, some high schools evidently were indicating neither class rank nor cumulative GPA of 
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any kind on transcripts (either that or the original data entry for the NELS high school transcripts 
did not compute GPA). While 13 percent of the HS&B/So students were missing the combined 
CLSRANKQ variable, half again of that proportion (20 percent) of the NELS students lacked a 
CLSSRNKQ value (see Glossary for further details and citations). 

This difference would have serious consequences in the replication of the original Tool Box 
hypotheses. The universe of NELS students with positive CLSSRNKQ values was initially 
smaller than those for the curriculum variable (HSCURRQ) and senior year test score 
(SRTSQUIN). Since the multivariate analyses of both Tool Box studies depend on positive 
values for all three variables, the composite variable built from a partition of the three would 
overestimate the value of CLSSRNKQ at the expense of the other two components. In preparing 
the analysis file for this study, plausible values for missing cases of CLSSRNKQ were imputed 
when values for both curriculum quintile (HSCURRQ) and test score quintile (SRTSQUIN) 
were positive and equal, e.g., both were in the third quintile. In these cases, CLSSRNKQ was 
assigned the same value as the other two variables. The impact of this imputation strategy 
lowered the unweighted missing cases of CLSSRNKQ from 20 to 17 percent, and the 
unweighted cases of NELS:88/2000 students missing one or more of the high school 
performance variables from 24 to 21 percent (see table Cl). 

Bias and potential imputation. 

Even after this revision, a problem remained, and a multiple imputation procedure was followed 
(Herzog and Rubin 1983). In the original edited version of the NELS high school transcript data 
that appears on NCES data CD 2003-402 (March 2003), and modified by its June 2004 
supplement, some 17.3 percent of the students were missing CLSSRNKQ, 13.8 percent were 
missing HSCURRQ, and 13.5 percent were missing SRTSQUIN. Under NCES statistical 
standards, CLSSRNKQ would have to be evaluated, since the proportion of students with non- 
missing values was less than 85 percent. 59 

The fonnula used to determine the nonresponse (i.e., missing) bias in these three variables was: 

A*(b-c) where A is the percent of students missing data, b is the percent of those 

missing data who did NOT earn a bachelor’s degree, and c is the percent of those 
not missing data who did not earn a bachelor’s degree. 

To obtain the relative bias with respect to the estimate, the result of this fonnula is divided by b. 

The initial nonresponse bias rates for the three variables were: 





Bias 


Relative bias 


HSCURRQ 


0.015 


0.021 


CLSSRNKQ 


0.034 


0.049 


SRTSQUIN 


0.039 


0.048 



59 See NCES 2002b. 
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In other words, for example, if the estimate for the curriculum variable was based only on those 
with positive values, it would overestimate the percent who did not earn bachelor’s degrees by 2 
percent. These are all small biases, and would not threaten tests of statistical significance. But 
the missing cases were less than random: Students who never entered postsecondary education 
by age 26 or 27 were overrepresented, and those who never graduated from high school or who 
were still enrolled in high school after the scheduled graduation of the class in 1992 would not 
show complete transcript records in 1992. 

In order to increase the number of students with positive and plausible values for all three 
components of the Academic Resources variable, another limited imputation was undertaken, 
focused on class rank/GPA. The first question identified, of all students who were in the 12th 
grade in 1992 and for whom the file showed positive values for both the curriculum 
(HSCURRQ) and test score (SRTSQUIN) variables: 



1) those who also showed a positive quintile value for class rank/GPA 

(CLSSRNKQ), and were in the same quintile on all three measures. 16.6% 

2) those who were in the same quintile on both curriculum and test score 

measures, also had a positive quintile value for class rank/GPA, but the rank of this 
quintile differed from the others by + 1 . 13.4 

3) those who were in the same quintile on both curriculum and test score 

measures, but were missing class rank/GPA. 2.7 

4) those who were missing class rank/GPA, and whose quintile positions 

on curriculum and test score measures differed by 1 . 2.5 

5) those who were in the same quintile on both curriculum and test score 

measures, also had a positive quintile value for class rank/GPA, but the rank of this 
quintile differed from the others by > 1 . 5.8 

6) those who were not in the same quintile on both curriculum and test score 

measures, whether or not class rank/GPA was missing. 59.0 



The focus of imputation was on those members of groups 3 and 4 whose high school transcripts 
registered 12 or more academic Carnegie units. This small group was then examined for prima 
facie anomalies in variables such as postsecondary attendance, selectivity of first institution of 
attendance (if any), and postsecondary remedial work. There were none. An algorithm was then 
developed that assigned a value to CLSSRNKQ, as follows: (1) where HSCURREV = 
SRTSQUIN, the class rank/GPA quintile was assigned the same value; (2) where 
the difference between HSCURREV and SRTSQUIN was +1, CLSSRNKQ was assigned the 
lower value in half the cases, and the higher value in the other half. 

The result reduced the proportion of missing (“nonresponse”) cases of CLSSRNKQ from 17.3 to 
15.8 percent. The nonresponse bias rates improved as follows: 

Before imputation After imputation 

Bias rate 0.034 0.019 

Relative bias 0.049 0.026 
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The consequent reconstruction of Academic Resources 

The proportion of students with non-missing values for all three components of the Academic 
Resources index rose from 70 to 72 percent as a result of the imputation described above. 
However small the increase, it required a recalculation of the comparative weights of the three 
components of Academic Resources, in the following steps: 

a) A logistic regression was run with bachelor’s degree attainment as the dependent 
variable, the three components as independent variables, and a universe limited to 1992 nth- 
graders. The ratios of the standardized beta coefficients for each of the independent variables 
produce the weights for each variable in the composite. The changes between the earlier version 
of these weightings and the post-imputation version are small, but required, as table C2 shows. 

Table C2. Differences in ratios of standardized betas for the three components of 

the Academic Resources composite, before and after limited imputation of 
missing class rank/GPA data for 1992 12th-graders 



Partition weight 

Component of Academic Resources Before imputation After imputation 

Curriculum 41.9 42.4 

Class rank/GPA 32.9 32.5 

Senior-year test score 25.2 25.1 

SOURCE: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402 and Supplement). 

b) Each of the three components is then multiplied by its weight brought forward from 
the logistic model. So, for example, a student in the third quintile of HSCURRQ is credited with 
1.272 points (3*0.424). The points are added, then weighted by the enhanced NELS high school 
transcript weight for those in the 12th grade in 1992 (F4F2HWT), then set out in quintiles of 
Academic Resources. The difference in distributions of the values of Academic Resources, 
before and after imputation, are indicated in table C3. 

Table C3. Differences in distribution of 1992 12th-graders across the values of the 
Academic Resources composite variable, before and after limited 
imputation of missing class rank/GPA data 



Percent distribution of students 



Values of Academic Resources 


Before imputation 


After imputation 


Missing 


29.9 


27.9 


Highest quintile 


14.3 


14.6 


2nd quintile 


13.3 


14.3 


3rd quintile 


14.2 


14.7 


4th quintile 


13.9 


13.6 


Lowest Quintile 


14.3 


14.9 



SOURCE: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402 and Supplement). 
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APPENDIX D 



Technical Issues 

Accuracy of estimates and standard errors 

There are different kinds of statistics in The Toolbox Revisited, and all of them are estimates 
derived from student samples. Two kinds of error occur when samples are at issue: errors in 
sampling itself, particularly when relatively small subpopulations (for example, American 
Indians) are involved; and nonsampling errors. Even in surveys as large as the three grade- 
cohort longitudinal studies used in this monograph, sampling errors can affect estimates of 
statistical significance. 

Non-sampling errors are more serious. A good example of a non-sampling error would be the 
fact that transcripts are missing for some students in all three grade-cohort studies. The 
transcripts are missing either because the student did not tell the interviewer that he or she 
attended the school (and there were no transfer credits on another transcript to identify the 
school); the school refused to send the transcript; the school could not find the transcript; the 
information sent by the school was not really a transcript; or while the student may have enrolled 
at the school he or she never registered for courses and did not generate a record. In this case, 
we can mitigate the effect of missing transcripts by differential weighting of the population, and, 
indeed, for both the High School and Beyond/Sophomore and NELS:88/2000 files, the analyst is 
given a choice of weights, one of which is confined to students with complete records (see the 
discussion of weights and flags below). Weighting, though, will not address the panoply of 
nonsampling errors. 

The effects of sampling and non-sampling errors ripple through databases. To judge the accuracy 
of any analysis, one needs to explicate and judge these effects. When the unit of analysis is the 
student, this is a straightforward issue because the original samples in the longitudinal studies 
consisted of students. For example, when questions are asked about the proportion of students 
who earned credits in an aggregate category of courses (e.g., table 21), the questions are about 
nonrepetitive behaviors of the students who were sampled. The descriptive comparisons in The 
Toolbox Revisited dealing with non-repetitive student behaviors require invocation of the 
Students’? statistic to determine whether the difference between two independent estimates is 
significant. The fonnula for computing Students’ t values is: 



(P1-P2) 

t = % (se^ + se 2 2 ) 

where Pj and P 2 are the estimates to be compared and sc, and se 2 are the corresponding standard 
errors. In this case, if t >1.96, one has a statistically significant difference at p <.05, a standard 
marker. For the judgments of statistical significance in all cross-tabulations in this document, an 
Excel template developed by MPR Associates for the production of reports to the National 
Center for Education Statistics, was used. 

The fonnula becomes more complex, however, for multiple comparisons among categories of an 
independent variable such as race/ethnicity. For multiple comparisons, the critical value for t 
rises depending on the number of comparisons that can be made in the family of the independent 
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variable. For race/ethnicity presented in five categories, there are 10 possible comparisons, so 
the significance level of each test must be p < .05/10 or p < .005. To determine the significance 
level of t values in any comparison of means or proportions, the result should be matched against 
standard published tables of significance levels for two-tailed hypothesis testing. 

When estimates are not independent, a covariance term must be added to the Students’ t formula 

(P .-Z-P^ 

t = %(se! 2 + se 2 2 ) -2(r)se 1 se 2 

where r is the correlation between the two estimates. The detennination of correlations requires 
a statistical software package, such as SAS or SPSS, and the invocation of proper weights for the 
comparison. 

Because none of the longitudinal studies invoked in The Toolbox Revisited was based on a 
simple random sample of students, the technique for estimating sampling error involves a more 
complex approach known as the Taylor series method. To produce Taylor series standard errors, 
the estimates presented used AM, a program developed by Jon Cohen and associates at the 
American Institutes for Research under contract to the National Center for Education Statistics. 

Flags and weights 

Each of the grade-cohort studies used in this monograph carries a complex set of flags and 
weights to mark the populations for which estimates are to be generated. The selection of these 
flags and weights is very important for both the accuracy and meaningfulness of estimates. 

For the postsecondary transcript sample of the High School and Beyond/Sophomore cohort 
(HS&B/So), the process was somewhat complex. Using the weights for the first follow-up 
survey (1982, the scheduled 12th-grade year for this cohort), three postsecondary transcript 
weights were developed. The first was based on a ratio of the sum of weights for all students in 
the 1982 panel who subsequently (in surveys of 1984, 1986 or 1992) claimed to have attended a 
postsecondary institution to the sum of weights for those for whom a transcript validating the 
claim was subsequently received. The ratio was then modified by factors derived from the 
stratification cells in the 1982 survey design to create multipliers that were applied to the raw 
weights for the students for whom transcripts were received or for whom postsecondary 
attendance was imputed from survey storylines. This is a generous formulation for all likely 
postsecondary participants. 

The second High School and Beyond/Sophomore weight involved the same procedure as the first 
but a more restrictive ratio applied to those students for whom a true postsecondary transcript 
was received. These students are more than “likely” participants; they are “known participants.” 
The third weight followed the same procedure as the second, but confined the population to only 
those students with complete postsecondary records (i.e., no missing transcripts). This weight is 
used in analyses of credit production and grades, since complete records are necessary for the 
analysis of both these features of student academic history. These weights are labeled PSEWT1, 
PSEWT2, and PSEWT3, respectively. 
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To accompany these weights for the comparisons that hold the population to students who were 
in the 12th grade in 1982, a special flag, SENRFLAG, was constructed from variables in the 
HS&B/So that described student status in 1982. Using the given flag for participation in the 
1982 cohort sample would be insufficient and not wholly accurate because not all students were 
in the 12th grade in 1982, e.g., students who graduated early from high school in 1981. But 
there were also students who were labeled “early graduates” on the data set (and thus candidates 
for exclusion from a 12th-grade flag) whose high school graduation date was listed as 1982. 
Early graduates were excluded, and erroneously labeled “early graduates” were included in the 
population with SENRFLAG = 1. If these students were not participants in the 1982 panel (even 
if postsecondary transcripts were received) their weight = 0. Using the 1982 panel weight alone 
without this flag will not produce an accurate universe of 1982 12th-graders. 

For all calculations of HS&B/So data in this document, SENRFLAG = 1, and the appropriate 
PSE weight invoked. 

The weights and flags for the NELS:88/2000 are more complex, still, because the cohort, 
established in the eighth grade, was “refreshed” twice: first, to be representative of the census of 
lOth-graders in 1990, and second, to be representative of the census of 12th-graders in 1992. 

The weights deriving from the 1992 12th-grade refreshing are at the core of weights 
subsequently developed for the postsecondary transcript sample. The same three postsecondary 
weight types developed for the High School and Beyond/Sophomores were employed here, but 
in combination with the 12th-grade (second follow-up survey, or F2) weight and the student’s 
presence in the final (2000) survey panel, F4. In addition, a set of weights based on the NELS 
high school transcripts in combination with the three postsecondary weight types also was 
developed when questions arise concerning the relationship between secondary school variables 
derived from high school transcripts and postsecondary variables derived from postsecondary 
transcripts. 

The most frequently used NELS:88/2000 weights in The Toolbox Revisited are: 

F4F2P2WT For all known postsecondary participants who were 12th-graders in 1992. 
F4F2HP3W For all postsecondary participants with complete records who were 12th- 
graders in 1992 and for whom high school transcripts are also part of the 
file. 

F4F2P3WT For all postsecondary participants with complete records who were 12th- 
graders in 1992. 

As in the case of the High School and Beyond/Sophomore cohort, a special flag was developed 
for 12th-graders in 1992. The existing flag on the NELS:88/2000 files excluded over 250 
students who, in fact, were awarded high school diplomas in the spring of 1992 and who carry 
positive weights for the panel (the descriptive windows of the Electronic Code Book for the 
fourth follow up survey of 2000 offer no reasons or clues for this anomaly). These students are 
included in the flag, GRADE 12 A, used in this monograph. 

The weighted Ns for samples used in a table are provided in the notes to the tables. Even if the 
same weight and flag is used on two tables, the weighted Ns may differ slightly because missing 
values in a particular variable are excluded from the calculations. 
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Multivariate analyses 

For all multivariate analyses in this monograph, special procedures were employed in 
accordance with the complex sampling designs of NCES longitudinal studies. These procedures 
are in the spirit, though not exactly to the letter, of those discussed and recommended by Thomas 
and Heck (2001). Thomas and Heck recommended alternative ways of producing both 
regression models and their adjusted standard errors in a single step, as opposed to the two-stage 
procedure used in this study. The reader should be aware that the software employed in this 
study, AM, correctly estimates standard errors associated with complex, cluster samples. 

For any model, an adjusted weight based on the population with non-missing values on all 
variables in the model was calculated, in the following steps: 

1. A weight appropriate to the question was selected. For example, for determinants 
of transfer from a community college to a four-year college, the weight for NELS 
students with received postsecondary transcripts, F4F2P2WT, was chosen. 

2. A simple tabulation of the dependent variable was then run for students who 
evidenced positive (non-missing) values for all variables in the model. Call this 
universe A. The selected weight was invoked. 

3. The log of the program for step 2 produces both unweighted and weighted Ns for 
universe A. 

4. The selected weight is then adjusted by what Thomas and Heck (2001) refer to as 
NORMWT, i.e., the weighted N / unweighted N of universe A. 

5. The selected weight is then multiplied by the NORMWT, thus adjusting the 
effective sample size. 

6. The example would look as follows: 

F4F2P2WT / (f4f2p2wtA / unweighted N for A configuration of variables) 

7. The result becomes a variable in its own right, a weight with a name, e.g., 
COREWT1. 

A root design effect (DEFT) reflects the effect of departures from simple random sampling, and 
in the methodology reports for the NELS:88/2000, is calculated for sub-populations on each of a 
selection of variables (Haggerty et al. 1996; Curtin, Ingels, Wu, and Heuer 2002). For any 
logistic model in The Toolbox Revisited, a DEFT based on the population with non-missing 
values on all variables in the model was calculated in order to adjust the standard errors 
produced by statistical packages such as SAS (used in the production of this study). The DEFT 
is calculated in three steps: 
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1 . The requisite data for a simple standard error are produced by the same equation 
used for NORMWT, and set out as follows: 



P(l-P) =s.e. 
r N 

2. A matching Taylor series standard error is produced by AM software with the 
same dependent variable and equation, with the population fdtered by positive 
(non-missing) values for the same variables used in the equation in step 1 . The 
Taylor series s.e. takes into account both stratum and primary sampling unit in 
combination with the weight selected, hence accounting for the complex sampling 
design. 

3. The DEFT = Taylor Series s.e. / simple s.e. 

Every discrete multivariate analysis has a unique DEFT. The DEFTs for the NELS:88/2000 are 
rather substantial, e.g., 1.83, reflecting not only the original sampling design in 1988 but also the 
successive "refreshings" of the sample in 1990 and (for the analyses in this monograph) in 
1992. 60 They are used to adjust the standard errors in the multivariate analyses, and hence 
reduce the likelihood of overestimating the effects of independent variables. The effect of the 
DEFT also is reflected in the production of the F and t statistics, for which the formulas used are: 



€ke. x DEFT° = F 
and 



_F_ 

r DEFT 2 = t 

In the logistic models employed in The Toolbox Revisited, the level of significance of the t 
statistic — p — for a two-tailed test is determined by reference to a standard table of critical values 
of t that can be found in any statistics textbook. 

Collinearity data 

This study includes Appendix J with tolerance statistics for collinearity for all the logistic model 
tables in order to assure the reader that the independent variables included in the models do not 
exhibit a nearly perfect linear relationship, in other words, that they do not overlap to such an 
extent that analysis of each variable, taken singly, is impossible. There are sometimes situations 
in which one variable is part of the definition of another, e.g., family income in relationship to 



60 Curtin, Ingels, Wu, and Heuer (2002) report the mean DEFT for all students in the fourth follow-up 
(2000) survey of the NELS as 1.954 (p. 209). The range of DEFTs in The Toolbox Revisited is 1.76 to 2.19. 
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socioeconomic status or multi-institutional attendance in relation to transfer. In cases such as 
these, collinearity statistics are produced as by-products of Ordinary Least Squares regressions 
for the variables in the logistic models. 

Of these by-products, tolerance is the statistic of choice: It measures how well one independent 
variable can be predicted by other independent variables in the model, and is equal to 1-R 2 for 
that variable. Ideally one would want this predictive relationship to be close to 1. The literature 
(e.g., Belsley, Kun, and Welsch 1980) suggests that there are no hard and fast rules for 
determining collinearity using either tolerance or its inverse, variance inflation, but that, in 
general, a tolerance reading of .20 or less evidences collinearity problems. This study adopted a 
tolerance threshold of 0.50, i.e., any reading above that threshold indicates no serious 
collinearity problem. Any indication below that point required further investigation of the 
independent variable(s) in question through correlation matrices. If the variables in question 
were part of the constructs of other variables, a choice was made as to which variable would be 
dropped. Dropping independent variables is one of the recommended options when potential 
multicollinearity arises (Knight 2004). 
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APPENDIX E 



Populations Included and Excluded From the Study Universe 

The population subject to analysis in The Toolbox Revisited consisted of 1992 12th-graders who 
subsequently attended a four-year college at any time through December 2000, who earned their 
high school diploma by December 1996, who presented complete high school transcript records 
and a senior year test score, whose postsecondary records were complete, and whose 
socioeconomic status was known. But there are other students who participated in the 1992 
NELS:88/2000 survey and who subsequently attended a four-year college at some time who are 
not part of this population principally because (a) one or more of these data elements were 
missing or (b) because they were not in the 12th grade in 1992. 

It is natural to ask about the demographic characteristics of the students who are excluded from 
the analysis because they are missing data or because they were not in the 12th grade in 1992, 
compared with the demographic characteristics of the students who are included. In order to 
compare distributions of this comparative demography, we include all 1992 survey participants. 

Considering those who attended a four-year college at some time but were nonparticipants 
because one or more of the key data elements were missing, key contrasts include: 

• A higher proportion of minority students than white students; 

• A higher proportion of students who were 20 years old or older in 1992 than those 
who were less than 20 years old; 

• A higher proportion of students from the lowest third of family income 
distribution than those from higher income levels; 

• A higher proportion of first-generation students than those whose parents had 
either some college or who had earned at least a bachelor’s degree; 

• A higher proportion of nonnative speakers of English than those from English 
monolingual backgrounds; 

• A higher proportion of those with three or more siblings than those who were 
either only children or had one or two siblings; 

• A higher proportion of those who became parents by age 20 than those who did 
not; 

• A higher proportion of those who had been retained in grade at least once than 
those who were never held back; and 

• A higher proportion of those who came from high schools in urban areas than 
those from suburban or rural areas. 

Some 60 percent of this group was missing only one of the key data elements, but it was a very 
important one: a high school transcript. 

Of nonparticipants because they were not in the 12th grade in 1992, we notice distinctively 
higher proportions by age (20 years or older), origin in a second language household (though not 
among nonnative speakers of English), and parental status, and among those who had been 
retained in grade and those who earned General Education Diplomas. 
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Table El. Percentage distribution of participants in the 1992 survey of the 

NELS:88/2000 by participation status in The Toolbox Revisited universe, by 
demographic and schooling background characteristics 



Characteristics 


Participants 
in the universe 
of this study 


Nonparticipants 3 
who were 

12th-graders in 1992 


Nonparticipants' 
who were not 
12th-graders in 


All students 


77.8(1.11) 


20.6(1.05) 


1.6(0.33) 


By gender 


Male 


76.5 (1.57) 


21.3 (1.46) 


2.2 (0.57) 


Female 


79.2 (1.35) 


19.8 (1.34) 


1.0 (0.22) 


By race/ethnicity 


White 


81.1 (1.08) 


18.0(1.07) 


0.9 (0.20) 


African-American 


64.3 (4.44) 


30.0 (4.04) 


5.7(2.72) 


Latino 


66.2 (4.11) 


29.6 (3.99) 


4.2(1.19) 


Asian 


73.2 (5.09) 


26.6(5.09) 


0.1 (0.09) 


American Indian 


53.6(12.8) 


43.1 (13.0) 


2.4 (2.30) 


By age in 1992 


Missing 


48.5 (9.15) 


51.5 (9.15) 


# 


20 or older 


41.6(8.56) 


47.9 (9.52) 


10.4 (4.86) 


Under 20 


79.2 (1.06) 


19.3 (1.01) 


1.5 (0.34) 


By family income 


Missing 


70.8 (3.02) 


27.1 (2.55) 


2.2(1.24) 


Highest third 


81.5 (1.44) 


17.3 (1.41) 


1.2 (0.37) 


Middle third 


80.1 (1.79) 


19.2(1.79) 


0.7(0.17) 


Lowest third 


71.3 (1.74) 


25.2 (2.60) 


3.5 (1.12) 


By postsecondary 
generational status 


Missing 


66.2 (4.08) 


33.4 (4.07) 


0.4 (0.24) 


First generation 


73.7 (2.73) 


23.8 (2.74) 


2.5 (0.68) 


Parents had some college 
Parents earned bachelor’s 


78.3 (1.72) 


19.2(1.60) 


2.5 (0.74) 


81.2(1.34) 


18.1 (1.32) 


0.7 (0.27) 


By second language 
background 


Nonnative speakers 


68.8 (3.78) 


28.9 (3.73) 


2.3 (0.81) 


From second language 


households 


68.3 (5.48) 


22.5 (4.94) 


9.2 (4.06) 


Monolingual English 


79.0(1.12) 


19.7(1.08) 


1.3 (0.35) 



See notes at end of table. 
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Table El. Percentage distribution of participants in the 1992 survey of the 



NELS:88/2000 by participation status in The Toolbox Revisited universe, by 
demographic and schooling background characteristics — continued 




Participants 


Nonparticipants 3 


Nonparticipants 3 




in the universe 


who were 


who were not 


Characteristics 


of this study 


12th-graders in 1992 


12th-graders in 1992 


By number of siblings 








None 


76.0 (4.37) 


23.3 (4.39) 


0.6 (0.31) 


One or two 


81.8 (1.15) 


17.2(1.13) 


1.0 (0.24) 


Three or more 


71.8(2.04) 


25.5 (1.87) 


2.7 (0.84) 


By parenthood 








Parent by age 20 


64.6(5.38) 


22.5 (4.07) 


12.9 (4.62) 


Not a parent by age 20 


78.3 (1.10) 


20.5 (1.07) 


1.2 (0.25) 


Held back in grade at 








least once 








Missing 


59.2 (6.33) 


38.5 (6.21) 


2.3 (1.11) 


Yes 


62.9 (4.85) 


30.4 (4.54) 


6.7 (2.64) 


No 


79.9 (1.07) 


18.9(1.04) 


1.2 (0.31) 


By urbanicity of high 
school location 








Urban 


69.9 (2.36) 


27.1 (2.15) 


2.9 (0.99) 


Suburban 


80.1 (1.58) 


19.0(1.57) 


0.9 (0.28) 


Rural 


83.9(1.71) 


14.9(1.71) 


1.2 (0.29) 


By type of high school 
diploma b 








Standard 


79.0(1.07) 


20.5 (1.06) 


0.5 (0.13) 


GED 


8.7 12.56) 


23.6 (6.49) 


67.7 (7.10) 


a Nonparticipants are defined as students who were either (a) missing data on senior year test score or 
socioeconomic status or both, (b) missing an in-scope high school transcript, (c) graduated from high school in 



January 1997 or later, (d) presented an incomplete postsecondary record, and/or (e) were not in the 12th grade in 
1992. 

b A tiny percentage who received a certificate of attendance was included with the GEDs. 

# Rounds to zero. 

NOTES: Standard errors are in parentheses. Universe consists of students who participated in the 1992, 1994, and 
2000 follow-ups of the NELS:88/2000 and who attended a four-year college at some time. Weighted N=1.45M. 
SOURCE: National Center for Education Statistics, NELS:88/2000 Postsecondary Transcript Files, NCES 2003- 
402 and Supplement. 
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APPENDIX F 



Gradations of Academic Intensity of High School Curriculum 

The following figure sets forth the 3 1 gradations (in descending value) of academic curriculum 
intensity and quality as used in the development of the Academic Resources index and variable 
for the NELS:88/2000 cohort. The figures in the boxes represent the minimum rounded number 
of Carnegie units required for the gradation on a given row. Where a cell is empty, there are no 
minimum requirements. Where a cell indicates "none" (for remedial math and remedial 
English), it means that no remedial work is allowed for that gradation. Where the cell for AP 
courses indicates zero, that means the student did not take any AP courses, not a minimum. For 
the NELS:88/2000 cohort, computer science was not nearly as widely offered as it is today. 
Therefore, computer-related credits were brought into play only to disaggregate lumps in the 
distribution. Total high school academic credits is an empirically-derived factor that comes into 
play only in the very lowest gradations. 

The basic five-subject credit thresholds were constructed in the course of examining the edited, 
coded transcript data for students who were known high school graduates with graduation dates 
through Dec. 3 1, 1996. The editorial process paid particular attention to all cases that showed 
less than 16 total high school credits. Where the evidence strongly suggested dissonance with 
other variables in the student’s record, all transcript records from that student’s school were 
examined. Where nonstandard credit metrics were found, they were adjusted with reference to 
state standards for high school graduation (Medrich, Brown, and Henke 1992), and major 
components (e.g., mathematics, English, etc.) multiplied or divided by as much as (but no more 
than) two. For example, when a group of students from the same high school showed 40-45 
Carnegie units in a state that required 20 for an academic diploma, the editorial process cut those 
40-45 units in half — across all subjects in which they were given. The editorial process also 
Windsorized cases of total Carnegie unit counts above 32, adjusting the major components down 
one-by-one, and dropped fragmentary transcripts with less than 6 Carnegie unit counts. 

As noted in the parallel appendix in the original Tool Box : 

These gradations of academic intensity and quality are based on the history of 
one national high school class that was scheduled to graduate in 1982. The next 
graduating class for which we possess similar data is that of 1992. While the 
specific number of Carnegie units, APs, and remedial indicators might change, 
the basic form and principles of the gradations will probably not change. This 
presentation of the possibilities of high school curricular attainment is criterion- 
referenced: theoretically, everybody can reach gradation level #1. (p. 1 14) 

The account of curriculum for the class of 1982 had 40 gradations. This account, for the class of 
1992, has 31. One implication of the shrinking number of gradations is that, in fact, more 
students were moving up the academic intensity ladder, clustering at higher criterion-referenced 
levels. 

Table FI presents the actual mean number of Carnegie units earned in core academic fields, 
irrespective of the theoretical thresholds, for students in each of the five quintiles of academic 
intensity derived from the 3 1 more detailed gradations. 



143 




Figure 



Grada- 

tion 



Curriculum components of the 31 gradations of the high school academic 
intensity measure of the NELS: 88/2000. bv Carneeie unit minimums 



Total 

Foreign Hist and Flighest Remed Remed Computer Academ 



>2.0 >2.0 >Alg2 None None >1 



>2.0 >2.0 >Alg2 None None 



>2.0 >2.0 >Alg2 None None 



>2.0 >2.0 >Alg2 None None 



English 


Math 


Science 


3.75 


3.75 


>2.0* 


3.75 


3.75 


>2.0 


3.75 


3.75 


>2.0 


3.75 


3.75 


3.0 


3.5 


3.0 


2.0 


3.5 


3.0 


2.0 


3.5 


3.5 


2.0 


3.5 


3.0 


2.0 


3.0 


3.0 


2.0 


3.5 


3.5 


2.0 


3.5 


3.5 


2.5 


3.0 


2.0 


1.0 


3.0 


2.5 


2.0 


3.0 


2.5 


2.0* 


3.0 


2.5 


2.0* 


3.0 


2.5 


2.0 


3.0 


2.5 


1.0 


3.0 


3.0 


1.5 


2.5 


3.0 


1.5 


2.5 


2.5 


1.5 


2.5 


2.5 


2.0 


2.5 


2.5 


1.0 


2.5 


2.0 


2.0 


2.5 


2.0 


2.0 


2.5 


2.0 


1.0 


2.5 


2.0 


1.0 


2.5 


2.0 


1.0 



2.0 



2.0 



2.0 



2.0 



2.0 



2.0 



2.0 



1.0 



2.0 



2.0 



2.0 



2.0 



2.0 



1.5 



1.5 



1.0 



1.0 



1.0 



1.5 



1.5 



0.5 



>Alg2 None None >1 



>Alg2 None None >0 



>Alg2 None None 0 



>Alg2 None None 0 



Alg2 None None >0 



Alg2 None None 0 



Alg2 None None 0 



>Alg2 None None >0 



>Alg2 None None 0 



>Alg2 None None 0 



Alg2 None None 0 



Alg2 None None 0 



<Alg2 None None 0 



<Alg2 



Alg2 None 



<Alg2 




Net 


0 


Net 


0 


Net 


0 


Net 


0 


Net 


0 


None 


Net 


0 
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28 


2.5 


1.5 


1.0 




0.5 




Net 1 










29 


2.5 


1.5 


1.0 




0.5 














30 


2.0 


1.0 


0.5 




0.5 














31 






















>6 



NOTES: (1) Net 1 means the sum of total mathematics credits minus remedial mathematics credits was 0.5 or less, 
i.e., if remedial math appeared at all on a student’s transcript, it was a major presence; Net 0 means the sum of total 
mathematics credits minus remedial mathematics credits was more than 0.5, i.e. of remedial math appeared at all on 
a student’s transcript, it was a minor presence. 

(2) The figures in the cells for English, math, science, foreign languages, and history and social studies 
represent the minimum rounded number of Carnegie units required for the gradation on a given row. Where a box 
is empty, there are no minimum requirements. 

(3) An asterisk in a cell for science credits indicates core laboratory science (biology, chemistry, and physics). 

(4) The reference points for highest level of mathematics studied in high school are higher than Algebra 2 
(>Alg2), Algebra 2 (Alg2), and less than Algebra 2 (<Alg2). Where there is no entry in the cell, there is no highest 
mathematics requirement for that row. 

(5) Minimum requirements for total high school academic Carnegie units, e.g., =>12 and =>6, come into play only 
in the very lowest gradations of the curriculum distribution. 

(6) When the distribution of students across these 3 1 levels is weighted and then aggregated to quintiles, the quintile 
breaks are as follows: 1-8 (highest quintile), 9-15 (2nd quintile), 16-20 (3rd quintile), 21-25 (4th quintile), and 
26-31 (lowest quintile). 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402). 

Table FI. Of 1992 12th-graders with complete high school transcripts, mean Carnegie 
units earned in core high school academic fields, percent of students whose 
highest level of high school mathematics was above Algebra 2, and mean 
number of Advanced Placement (AP) courses, by quintile of academic 
curriculum intensity 



Core high school academic curriculum fields 



Academic 

curriculum 

intensity 

quintile 


English 


Math 


Core lab 
science 


Foreign 

lan- 

guages 


Flistory 

and 

social 

studies 


Com- 

puter 

science 


Percent with 
highest math 
above 
Algebra 2 


Total AP 
courses 


Highest 


4.27 


4.10 


3.20 


3.09 


3.70 


0.74 


96.4 


0.644 


2nd 


4.17 


3.81 


2.71 


2.23 


3.62 


0.56 


64.7 


0.068 


3rd 


4.23 


3.11 


1.99 


1.98 


3.47 


0.59 


0 


0.003 


4th 


4.10 


2.98 


1.36 


0.74 


3.44 


0.61 


0.71 


0.019 


Lowest 


3.43 


1.81 


0.94 


0.62 


2.82 


0.28 


0.05 


0.006 



SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402). 
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APPENDIX G 



Logistic Models for Two Alternative Presentations of 
High School Academic Background 

Table Gl. Logistic account of factors associated with earning a bachelor’s degree in the 
history of 1992 12th-graders who attended a four-year college at any time, 
version 2: Demographic and high school background using the three 
components of Academic Resources as discrete variables 



Adjusted 
Parameter standard 

Variable estimate error t v Delta-p Tolerance 3 



Intercept 


-4.7580 


0.6530 


3.33 


0.02 






Curriculum quintile 


0.3782 


0.0738 


2.34 


0.05 


0.0878 


0.6729 


Class rank/ GPA quintile 


0.3886 


0.0712 


2.49 


0.05 


0.0899 


0.6268 


Senior test score quintile 


0.0581 


0.0754 


0.35 


t 


t 


0.6079 


Socioeconomic status quintile 


0.3075 


0.0638 


2.20 


0.10 


0.0712 


0.8632 


Education expectations 


0.5351 


0.2020 


1.21 


t 


t 


0.8949 


Race 


-0.5191 


0.2020 


1.17 


t 


t 


0.8757 


Gender 


-0.4076 


0.1525 


1.22 


t 


t 


0.9477 


Parenthood 


-1.6210 


0.4834 


1.53 


t 


t 


0.9721 



t Variables did not meet threshold criterion for statistical significance 
a For details on Tolerance, see Appendix D. 

NOTES: Statistically significant variables are highlighted in bold. Standard errors adjusted by design effect = 2.17. 
G 2 = 5206.65; df = 4943\ G 2 /df= 1.0529; X 2 (df) = 33.78 (8); pseudo R 2 = 0.227; percent concordant predicted 
probabilities = 78.9. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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Table G2. Logistic account of factors associated with earning a bachelor’s degree in the 
history of 1992 12th-graders who attended a four-year college at any time, 
version 3: Demographic and high school background using three proxy 
variables for high school academic curriculum intensity 



Variable 



Adjusted 
Parameter standard 
estimate error 



P 



Delta-p 



Tolerance 3 



Intercept 


-5.0792 


0.6795 


3.41 


0.01 






Advanced placement 


0.8931 


0.2715 


1.50 


t 


t 


0.8462 


Science momentum 


0.4005 


0.0990 


1.85 


0.10 


0.0927 


0.6341 


Foreign language 


0.1163 


0.0708 


0.75 


t 


t 


0.7843 


Class rank / GPA quintile 


0.3317 


0.0703 


2.15 


0.10 


0.0768 


0.6475 


Socioeconomic status quintile 


0.2957 


0.0638 


2.12 


0.10 


0.0685 


0.8597 


Education expectations 


0.5305 


0.2055 


1.18 


t 


t 


0.8960 


Race 


-0.4740 


0.1963 


1.10 


t 


t 


0.9234 


Gender 


-0.4022 


0.1569 


1.17 


t 


t 


0.9036 


Parenthood 


-1.6042 


0.4838 


1.51 


t 


t 


0.9730 



■j" Variables did not meet threshold criterion for statistical significance 
a For details on Tolerance, see Appendix D. 

NOTES: Statistically significant variables are highlighted in bold. Standard errors adjusted by design effect = 2.19. 
G 2 = 5193.69; df= 4939- G 2 / df= 1.0515; X 2 (df) = 33.94 (9); pseudo R 2 = 0.229; percent concordant predicted 
probabilities = 79.2. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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APPENDIX H 



The Timing of Departure 

When this narrative took up the topic of the second calendar year of student histories and pointed 
out that roughly one out of five of our subjects had become status dropouts by that point, the 
question naturally arose: What happens to all those students who leave after the first year — or 
even the first semester — when most attrition occurs? How does one account for them? We need 
to provide some corrective data in this excursion, since we considered status dropout status only 
in the context of the paradoxes of first-to-second year persistence. 

With rare exceptions (e.g., Tinto 1987), most attrition studies are (a) short-tenn, as in first year 
to second year, without allowing for the fact that students who skip the second year may return 
in the third or fourth years, and (b) limited to attrition at the same institution, without allowing 
that the student who can no longer be found at Old Siwash may, in fact, be enrolled at Greentree 
Valley Community College. Students who leave one school and attend another one are not 
dropouts — they are most likely transfers. Tinto (1987) made that point a long time ago, and it 
should be obvious. If they are not transfers, they may be on temporary "excursions" or more 
sustained voyages that end in either fragments or "discovery" (Adehnan 2004b). 

Given the systemwide perspective and the length of a longitudinal study such as the 
NELS:88/2000, a more sophisticated question about withdrawal can be asked, looking 
backwards from the last month of the longitudinal study (December 2000): 

What percentage of traditional-age students who entered postsecondary education 
during this period are no longer enrolled — anywhere — and never completed a 
credential of any kind, by timing of the gap between their first month and last 
month of enrollment? 

This question follows the student — wherever the student goes. It tells us who became a “status 
dropout” by age 26 or 27 (which does not preclude even these students from returning at a later 
moment). And it tells us when the student became a status dropout, setting up an inquiry 
concerning the reasons for departure that follows Tinto’s (1988) common sense suggestion that 
these reasons change by point of departure. Table HI sets forth these data for all 1992 
12th-graders who entered postsecondary education, by type of institution first attended. 

Yes, the highest attrition rate in postsecondary education occurs in the first year (Schutz and 
Malo 2003), but table HI shows that the proportion of traditional-age students who leave in the 
first year and never return is not radically higher than the proportion of those who leave at later 
points in time, and differs by type of institution first attended. Consider the face validity of these 
data: The mean number of credits earned by students who ultimately became status dropouts, by 
period of departure, matches roughly what one would expect — half or less of the full-time norm 
for the period. Proportion, however, is not the ideal way to assess the phenomenon. 

Event-history accounts of attrition offer a more enlightening approach (e.g., Ronco 1996; 
DesJardins, Ahlburg, and McCall, 1999). The driving force of event history is risk or hazard, as 
befits a methodology derived from medical history analyses (Singer and Willett 1991), as in: 
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given the patient’s medical momentum and other configurations of the patient’s life-style, 
demography, and environment, when is the patient at risk of dying (the ultimate censoring event) 
if the patient did not die last year? The flip side of risk in event history is survival. While the 
purpose of this study stops short of survival analysis, the first step involved in setting up risk 
modeling involves detennining hazard probabilities, and these are instructive in light of the 

Table HI. Percentage of 1992 12th-graders who entered postsecondary education and 
withdrew without completing any credential by December 2000, by timing of 
withdrawal and institution of first attendance 



Institution of First Attendance 



Withdrawal behavior 
and its timing 


Four- year 


Community 

college 


Other 

sub-bacc a 


Mean 

credits 


Percent of 

all 


Left postsecondary without a 
credential and did not return 












Left within 1 1 months of 
first enrollment 


4.6 (0.59) 


14.6(1.13) 


15.5 (2.77) 


9.85 (0.54) 


8.9 (0.58) 


Left within 12-23 months 


3.3 (0.41) 


11.0(1.32) 


6.5 (1.50) 


21.64 (0.99) 


6.7 (0.59) 


Left within 24-47 months 


4.9 (0.41) 


12.5 (1.11) 


4.6(1.05) 


39.88 (1.62) 


7.9 (0.50) 


Left within 48-102 months 
and not enrolled in 2000 


8.7 (0.73) 


14.3 (1.23) 


4.0(1.62) 


58.29 (2.68) 


10.6 (0.65) 


No credential but still 
enrolled in 2000 


4.7 (0.52) 


9.2 (0.94) 


1.8 (0.85) 


70.98 (3.56) 


6.4 (0.47) 


Earned credential 


73.8 (1.09) 


38.4(1.67) 


67.7 (3.55) 


124.73 (0.90) 


59.5 (1.05) 



a Sub-baccalaureate institutions other than community colleges. 

NOTES: Standard errors are in parentheses. Columns may not add to 100 percent due to rounding. "Credentials" 
can be certificates, associate degrees, or bachelor’s degrees. 

SOURCE: National Center for Education Statistics: NELS:88\2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 

first-year performance variables used in our analysis. That is, if we start at the first month of the 
first term of attendance and, for each calendar year following that point, asked what percentage 
of students left the postsecondary system with no degree and had not returned to school by 
December 2000, changing the denominator for each year to the number of students who 
remained in the system, we have a "declining risk set" (Ronco 1996) that serves as a rough 
guideline for the timing of potential dropout. The annual metric is obviously more even than the 
time brackets chosen for table HI. 
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The "declining risk set" for this study is presented in table H2. To summarize: For all 
postsecondary students, the risks of becoming a status dropout are highest in the first year, 
decline through the fourth year, flatten out through the sixth year, and decline after that. For 
students who attended a four-year college at any time, the risks of becoming a status dropout are 
much lower across the temporal board, are slightly higher in the first year, drop and then flatten 
from the second year through the sixth, and drop again after that. This is only the first step — and 
hardly the final analysis — of event history, but is perhaps a more accurate way of describing the 
parameters for understanding the phenomenon of when students leave postsecondary education. 

Table H2. Declining "hazard probabilities" for 1992 12th-graders who entered 
postsecondary education, by calendar year following first date of 
enrollment 



"Hazard probabilities" for leaving without a degree 







All who attended a 


Departure timing for those 




four-year institution 


who never returned by 
December 2000 


All students 


at anv time 


First calendar year 


.102 


.039 


Second calendar year 


.061 


.029 


Third calendar year 


.048 


.030 


Fourth calendar year 


.039 


.027 


Fifth calendar year 


.037 


.027 


Sixth calendar year 


.041 


.029 


Seventh calendar year 


.031 


.025 


More than seven calendar vears 


.026 


.020 



SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 

The reader of table HI will note that community college beginners exit sooner than do students 
who started at four-year colleges, and are more likely to still be enrolled at the end of the 
longitudinal study period. While there is comparatively little variance in the pace of pennanent 
dropout for students who started in community colleges when the periods between the first and 
the fourth year are considered, there is no question that beyond that point, attrition slows. The 
same cannot be said for the pace of status dropout for students who started in four-year colleges: 
The rate of permanent attrition appears to be stable over time, a finding slightly different from 
that of the "declining risk set" approach, and a product of the clumping of years in table HI. As 
for sub-baccalaureate institutions other than community colleges, which specialize in short-term 
certificate programs, the status dropout rate drops dramatically from the first year to the second 
and subsequent periods. 

Advancing on Tinto’s (1988) suggestion, Eaton and Bean (1995) hypothesize that the factors 
affecting attrition are different for first-year students who withdraw from higher education versus 
the factors that come into play for students who leave at later points in time. Unfortunately, 

Eaton and Bean could not investigate that hypothesis because their study was confined to first- 
year and second-year students at one institution (a public research university), and bracketed 
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only one year’s account of retention/attrition (for a similar inquiry, see Patrick 2001). The 
NELS: 88/2000 data, on the other hand, offer 8.5 years and a myriad of institutional types in 
which to explore and detail the boundaries of the issue. As Ronco (1996) notes, the timing and 
duration of "enrollment events" is a critical framework for identifying factors associated with 
graduation, transfer, and stop-out — the three modes of "exit" from an institution. 

Table H3 provides a framework for future consideration of the differential attrition hypothesis. 
The literature on the putatively sophisticated economic decisionmaking of adolescents with 
respect to entering higher education, even in the face of uncertainty of returns (Altonji 1993), 
can be extended to reasons for leaving a path to postsecondary credentials. As Beattie (2002) 
points out, human capital theory is not rigid: it recognizes something called individual taste or 
proclivity that may be only tangentially related to group membership (e.g., ethnicity, gender, 
family income). At the least, students already in the postsecondary system know that they will 
eventually be better off for their education efforts (or they wouldn’t have been there in the first 
place). So, what did the students in our universe who did not complete a bachelor’s degree 
program (and who were not still enrolled as degree candidates at the end of the 8.5 year period) 
tell us of their reasons for leaving, by period during which they became status dropouts? 

Table H3. Reasons for leaving postsecondary education without credentials by timing of 
exit: 1992 12th-graders who entered postsecondary education by December 
1996 



Percent offering these reasons for leaving postsecondary education 



Timing of 
withdrawal 


Finances 


Job/ 

Military 


Academic 


Personal/ 

Family 


Mood/ 

Lifestyle 


Left postsecondary 
without a credential 
and did not return 












Left within 1 1 
months of first 
enrollment 


19.7(3.10) 


14.8 (2.03) 


5.9(1.92) 


31.5 (3.51) 


25.9(3.99) 


Left within 12-23 
months 


25.2 (5.19) 


11.0(2.81) 


5.6(1.40) 


42.6(6.19) 


12.7(2.23) 


Left within 24-47 
months 


19.4 (3.60) 


17.4 (2.69) 


3.6 (0.99) 


35.2 (3.78) 


20.5 (2.30) 


Left within 48-102 
months and not 
enrolled in 2000 


17.8(2.19) 


18.9(2.45) 


7.7 (2.46) 


29.3 (4.04) 


21.2 (3.21) 



NOTES: Rows will not add to 100.0 because (a) student responses to questions about reasons for departure are not 
mutually exclusive, and (b) a category for "other" is not included. Standard errors are in parentheses. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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When the NELS:88/2000 students were asked (in the spring of 2000) why they left school, they 
were offered a list of reasons (though only 1 7 percent of students responding to the question 
indicated more than one reason). As table H3 indicates, personal and family reasons for 
departure dominate all other reasons for students who left postsecondary education without a 
credential between the 12th and 47th month following the first month of enrollment. The same 
factor ranks at the top versus all other factors except mood/lifestyle for students who left in the 
first year and students who left after the fourth year. Strictly financial reasons are not as 
prominent in students’ judgment as some contend. "Job/military considerations" might be 
considered "financial," but there is too much ambiguity in that category of response to leap to 
that conclusion. Some 6.6 percent (s.e. = 0.47) of thel992 12th-graders subsequently served in 
the military, but 14.9 percent (s.e. = 4.95) of status dropouts who cited financial reasons for 
leaving school had served in the military, and an identical 14.9 percent (s.e. = 3.84) who cited 
"job/military considerations" for leaving had served in the military. If the NELS files had 
offered a more complete labor market history, we might be able to resolve the ambiguities. 

As a partial confirmation of these observations, consider the work of Li and Killian (1999), who 
surveyed 622 students of all ages and levels who left a large state flagship university in 1997 
(present in the winter term but not the following fall) and elicited 45 discrete reasons for 
departure. First, Li and Killian discovered that 20 percent of these students were not really 
dropouts at all, rather either transfers to other institutions (principally for reasons of academic 
program or location) or stop-outs with intentions of returning. Of the remaining students, 43 
percent left for academic reasons (e.g., didn’t like the program, classes too large, poor 
performance), 34 percent left for personal reasons (e.g., unstated personal problems, location 
issues including homesickness, illness, “social” reasons), and 20 percent for financial reasons. 

Of the financial reasons, Li and Killian advise us that “individual patterns of money 
management, more than family income” may be responsible for financially-driven attrition 
(p. 12), and that if we are going to understand attrition at all so that student affairs officers can 
establish realistic efforts to address what is within the control of the institution to assist, we have 
to dig below the level of large-scale concepts of academic and social integration. There is a 
considerable range of reasons that students leave an institution, and, as Li and Killian point out, 
students usually have more than one reason for departure. We can’t control homesickness; we 
can help students deal with money management. 

As for academic reasons for departure, table H4 reveals that compared to those who earned 
bachelor’s degrees, no one in the departure universe was performing very well, no matter when 
they left, and even though a tiny percentage (see table H3) cited academic reasons for walking 
away. Braxton, Brier, and Hossler (1988) argue that when students are asked about their reasons 
for withdrawal in post-hoc surveys, they offer "socially acceptable rationalizations." We should 
hence be appropriately wary of the percentage distributions in table H3. Given the low 
percentage of students who cited academic reasons, and GPAs and cumulative credits for the 
same students that can be described as, at best, marginal, there is some indirect support for this 
contention. 
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Table H4. Mean postsecondary GPAs of 1992 12th-graders who withdrew without 
completing any credential, by timing of withdrawal, and compared with 
GPAs of those who had not completed a credential but were still enrolled 
in 2000, and those who earned bachelors degrees by December 2000 



Mean grade point average 



Departure behavior 
and tirnina 


In first calendar 
year of 
attendance 


Through second 
calendar year 
of attendance 


Entire 

undergraduate 

record 


Left postsecondary without 
a credential and did not return 








Left within 1st 1 1 months 
of first enrollment 


2.10(0.091) 


2.10(0.091) 


2.10(0.091) 


Left within 12-23 months 


2.30 (0.057) 


2.08 (0.058) 


1.83 (0.100) 


Left within 24-47 months 


2.31 (0.049) 


2.26 (0.046) 


2.03 (0.060) 


Left within 48-102 months 


2.10(0.059) 


2.02 (0.054) 


2.03 (0.050) 


No credential, but still enrolled 


2.14 (0.069) 


2.08 (0.069) 


2.19(0.078) 


Earned bachelor’s degree 


2.89 (0.015) 


2.91 (0.014) 


3.04 (0.011) 



NOTE: Standard errors are in parentheses. 

SOURCE: National Center for Education Statistics: NELS:88\2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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APPENDIX I 



Course Categories Included in Aggregates for Participation Rates 
In the First Two Calendar Academic Years of Attendance 



College-level writing 



Oral communications 
Computer-related 



Introductory biological 
sciences 

Introductory physical 
sciences 



Course categories 

English composition; technical writing; creative writing; advanced 
essay 

Oral communications; public speaking/debate; voice 

Introduction to computer science; introduction to computing; 
computer programming: general 

General biology; human biology; cellular biology; zoology: 
general; botany: general 

General chemistry; general physics; general geology 



College-level mathematics College algebra, liberal arts mathematics, finite mathematics, 

statistics, precalculus, calculus 

Core history Western civilization, world history, U.S. history surveys, European 

history 1789-present, Asian history, African history, Latin- 
American history 

General psychology [single course category] 

Micro/macroeconomics [single course category] 

Humanities other than Humanities: general; introduction to philosophy; ethics 

literature and foreign (philosophy); religious studies surveys; interdisciplinary 

languages humanities; arts/humanities 

Literature Literature: general; introductions to poetry, fiction, drama, non- 

fiction prose; English literature; American literature; Afro- 
American literature 



Core social sciences Anthropology: general; cultural anthropology; geography: general; 

introduction to political science; U.S. government; introduction to 
sociology 

Visual/graphic arts Visual communications; introduction to design; fine arts: 

introduction; art history; drawing 

Foundation business General business principles; business law/legal environment; 

introduction to accounting 
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APPENDIX J 

Collinearity Statistic (Tolerance) for the Seven Steps of the Logistic Narrative 

Variable Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 Step 7 

table 12 table 13 table 15 table 16 table 24 table 26 table 27 



Academic Resources quintile 


0.8618 


0.7454 


0.5672 


0.5638 


0.5631 


0.5554 


0.5957 


Socioeconomic status quintile 


0.8761 


0.8638 


0.8505 


0.8494 


0.8452 


0.8851 


0.8893 


Race/ethnicity 


0.9198 


0.9231 


0.9068 


0.9065 


0.8977 


C 


c 


Gender 


0.9712 


0.9623 


0.9404 


0.9403 


0.9351 


0.9187 


0.9184 


Parenthood by age 20 


0.9734 


0.9653 


0.9443 


0.9441 


0.9409 


0.9409 


0.9409 


Education anticipations 


0.9071 


0.8930 


0.8517 


0.8515 


0.8460 


0.8458 


0.8468 


Selectivity of first institution 


a 


0.8296 


0.8342 


0.8330 


0.8143 


0.8233 


0.8195 


No delay of entry 


a 


0.9034 


0.8663 


0.8599 


0.8633 


0.8698 


0.8664 


Acceleration credits 


a 


0.8862 


b 


b 


b 


b 


b 


Low credits in first year 


a 


a 


0.7645 


0.7611 


0.7125 


0.7096 


0.6860 


First-year grades 


a 


a 


0.8427 


0.8410 


0.8246 


0.7619 


0.7646 


First-year remediation 


a 


a 


0.7586 


0.7585 


0.7462 


0.7759 


C 


First-year college-level math 


a 


a 


0.7439 


0.7435 


0.7688 


d 


d 


Work-study 


a 


a 


a 


0.9563 


b 


b 


b 


Multiple institutions 


a 


a 


a 


a 


0.6671 


0.7976 


0.6601 


Classic transfer 


a 


a 


a 


a 


0.7216 


0.7985 


0.7049 


Four-to-four transfer 


a 


a 


a 


a 


0.7801 


0.9426 


0.7731 


Summer-term credits 


a 


a 


a 


a 


0.8926 


0.9030 


0.8978 


Ever part-time 


a 


a 


a 


a 


0.8192 


0.8056 


0.7130 


Trend in grades 


a 


a 


a 


a 


a 


0.9093 


0.9136 


Cumulative college math 


a 


a 


a 


a 


a 


0.8166 


0.8089 


Continuous enrollment 


a 


a 


a 


a 


a 


a 


0.7595 


Withdrawal/repeat ratio 


a 


a 


a 


a 


a 


a 


0.7816 



a Variable not included at this step of the logistic narrative. 

b Variable did not qualify to be carried forward to the next step of the logistic narrative. 
c Variable did not meet criterion for entry into the model at this step of the logistic narrative. 
d Replaced by cumulative college math in these steps of the logistic narrative. 

NOTE: For definition of tolerance and discussion of collinearity, see Appendix D. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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APPENDIX K 



So They Earned a Degree! Now, How Long Did It Take Them? 

The second most frequently asked question about U.S. higher education concerns time to degree. 
Despite accumulating evidence over the past quarter-century, the normative benchmark for 
completing a bachelor’s degree has remained at four elapsed academic years. Our ideal seems 
not only to get it over with, but to get it over with in normative time. An altruistic argument in 
favor of the nonnative is that when students finish in a timely manner, they make room for 
others, i.e., completion keeps the access channels open. Yet recipients of bachelor’s degrees are 
adults who have made a variety of decisions concerning competing goals and obligations that 
often do not include finishing a degree in four years. For example, if one looks at all recipients 
of bachelor’s degrees in 1999-2000, and asks when they commenced their studies and how old 
they were when the degree was awarded, one finds that 62.8 percent (s.e. = 1.5) of the 7.5 
percent of bachelor’s degree recipients who were over 40 in 2000 started out in postsecondary 
education before 1980. At a minimum, the elapsed time to degree for these people was 20 years 
(in fact, the average was over 27 years). Of the 9.2 percent (s.e. = 0.4) of degree completers who 
were between 30 and 39 in 2000, 73.3 percent (s.e. = 1.7) entered postsecondary education prior 
to 1990, so, at a minimum, the elapsed time to degree for these people was 10 years (in fact, the 
average was nearly 15 years). 61 

The key phrase for our purposes is "elapsed time," as that is the customary way of measuring 
time to degree. It is not surprising that the older the baccalaureate recipient, then more likely 
that student had taken at least two years off from higher education, and the more likely that their 
principal reason for doing so was "change in family status," as in becoming a parent and raising 
children. The reader might consider, in passing, when complaints are raised about excessive 
time to degree, whether our system of higher education should be "blamed" for providing the 
opportunity for adults with different life trajectories to return to school and complete degrees, 
even if the time between initial enrollment and degree award is 15 or more years. That is the 
practical meaning of elapsed time. 

There is an obvious difference between elapsed and enrolled time. When DesJardins, Ahlburg, 
and McCall (2002) investigated factors leading to "timely graduation," they confined one of their 
universes to students who had never stopped out — not even for the one term this study allows 
stop-out without labeling the student a non-continuous enrollee. 62 If we followed this mode of 
analysis, we could focus in on student uses of enrolled time to determine what contributed to and 
what detracted from year-by-year progress. Knight (2002) includes types and timing of 



61 All data in this paragraph were generated from the National Center for Education Statistics: 
Baccalaureate and Beyond Longitudinal Study, 2000: Data Analysis System. 

62 The research here involved a "competing risks" model in which the histories of students who had never 
stopped out were compared to the histories of those who had stopped out, hence, embraced both elapsed and 
enrolled time. 
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remediation, majors and changes-of-major, course withdrawals and repeats, course failures, 
credit hours earned and grades in general education courses, hours per week working and 
studying, and first-year grade point average among variables reflecting different aspects of the 
uses of time. 63 Volkwein and Lorang (1996) took up the phenomenon of students with extended 
time-to-degree because they purposefully carried lighter credit loads. Why did they carry less 
than the 15 credit per semester norm for the institution that served as the site for this study? To 
enhance GPA (an objective which precipitated dropping difficult courses), and to generate more 
free time, some of which was for work or family responsibilities, or both. Some respondents 
also cited difficulty of enrolling in a course "at the time I wanted," a phenomenon familiar to 
anyone who has worked student registration lines. 

We also know that many disciplines require credit hour production that, when translated into 
standardized semester metrics, exceed the 120 credits that would be produced by students who 
attended only during the regular academic year and carried a standardized full-time load of 15 
credits per semester. In a survey of 91 public universities in all 50 states, Pitter, LeMon, and 
Lanham (1996) found combinations of university and program requirements ranging from 
122-124 credits in social sciences, foreign languages, psychology, mathematics, and protective 
services, for example, to 130-142 credits in such fields as engineering, architecture, and health 
professions (e.g., nursing, physical therapy, etc.). They also confirm our observation of five-year 
bachelor’s degree programs in phannacy, for a noted case, with a median credit requirement of 
161. Any requirement above 120 credits will either add time to degree or encourage students to 
(a) earn credits by examination, including Advanced Placement, (b) attend during summer terms, 
and/or (c) carry credit loads in excess of 15 per semester in order to graduate "on time". 64 
As Garcia (1994) observed, whether the student is a community college transfer or a native 
student in a four-year college, the “basic recipe" for completing a bachelor’s degree within the 
standard "templates for time-to-degree is to maintain continuous enrollment and to earn more 
than 30 semester units each academic year" (p. 8). Sounds simple, but the list of diverting 
phenomena is long, complex, and often best assessed for indirect effects, as Garcia has done, 
using path analysis. 

What we learn from these studies of the uses of enrolled time in relation to normative models of 
time to degree is what to include in both descriptive and multivariate accounts of elapsed 
time — and elapsed time is the measure used in both the original Tool Box and The Toolbox 
Revisited. The descriptive statistics are offered to the reader in Appendix L, table LI 3, but some 
basic observations are worth repeating here. First, the mean elapsed time to degree for 
bachelor’s recipients in the NELS:88/2000 was 4.58 calendar years, and the median time (which 



63 Knight’s list for a “hypothesized model of effects upon total semesters elapsed” (Knight 2002, p. 6) also 
includes provisions for variables derived from student surveys that are parallel to those used in the National Survey 
of Student Engagement, e.g., in Kuh et al. 2001. 

64 The small number of NELS:88/2000 bachelor’s recipients who completed the degree in three elapsed 
calendar years or less entered with a mean of 10.6 acceleration credits (s.e. = 2.81) and subsequently accumulated a 
mean of 12.2 credits during summer terms (s.e. = 1.94). 
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is probably more meaningful) was 4.24 calendar years. 65 Shorter time to degree is observable 
among students from the highest quintile of high school Academic Resources, students who first 
entered highly selective institutions, among those who brought more than four acceleration 
credits across the matriculation line, and among those who attended only one school. 
Considerably longer time to degree is observable among those who attended three or more 
schools, among both community college to four-year college and four-year-to-four-year transfer 
students, among those who took more than one remedial course, those who were not 
continuously enrolled, and those who majored in the physical sciences. Most of these 
relationships confirm common sense. 

But it is the multivariate analysis that will tell us what counts, and that is the purpose of table 
Kl. Table K1 presents a linear (Ordinary Least Squares) regression in which the dependent 
variable, time to degree, is set on a scale with four values: more than 6 calendar years, 5-6 years, 
4-5 years, and less than 4 calendar years (which encompasses the normative four-year degree). 
With one exception, a dichotomous variable indicating whether the student ever engaged in a 
cooperative education or internship course, the independent variables selected were those 
previously explored and (in most cases) used in the logistic narrative of Part IV of this study. 

The statistically significant independent variables are highlighted in bold. How do we read the 
table? What do we see? 

On reading the table: First, the R 2 indicates that this linear model accounts for half the variance in 
timing of bachelor’s degree completion for the NELS:88/2000 cohort. That’s a very convincing 
level. Second, the positive parameter estimates mark variables that add time to degree, while the 
negative parameter estimates indicate variables that shrink it. Third, the following variables did 
not meet the significance threshold for entering the model (set at p < .05): race/ethnicity, 
becoming a parent by age 20, multi-institutional attendance, and freshman year remediation. 

There is no question of the major contributors to extended time to degree: 

1 . Excessive no-penalty course withdrawals (W) and no-credit repeats (NCR) — 
Damaging to degree completion, these are just as damaging to time to degree. 

Yes, we can keep students continuously enrolled, but if much of that enrollment is 
nullified by withdrawals and repeats, we have what Tinto calls retention without 
education. It’s worse, though, because every seat at every available hour in every 
facility of an institution of higher education is at stake, and every seat marked 
with a W or NCR bars another student from sitting down. When the blocked 
seats reach a critical mass, general access is impeded as well. This observation is 
worth repeating. 



65 Broh (1991) presented an intriguing argument for an alternative time to degree calculation that began 
with the median, not the mean, and included students who had not finished the degree but who were still enrolled as 
degree candidates. In The Toolbox Revisited, the median for students who had completed degrees is 4.24 calendar 
years. Including students who had not earned degrees by December 2000 but who were still enrolled as bachelor’s 
candidates, the median time to degree would be 4.49 — very close to the mean of 4.58. 
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2. Transfer, both classic community college to four-year, and four-year to four- 
year — As previously noted, transfer often involves a hiatus in enrollment and 
extra effort to meet prerequisite requirements at the new institution. 

3. Change of major — Garcia (1994) observed that the more times the student 
changed major, the greater the elapsed time to degree. Knight (2002, 2004a, 
2004b) also includes multiple changes of major among the set of variables with 
significant impact on both enrolled and elapsed time. 

4. Whether the student was ever part-time — Since our variable for part-time 
attendance was constructed partly with reference to course withdrawals that, in 
effect, produced lighter course loads, the result here backs up the conclusions of 
Volkwein and Lorang (1996). 



Table Kl. 



Ordinary Least Squares regression indicating factors influencing time-to- 
degree for 1992 12th-graders who earned bachelor’s degrees by December 



2000 



Variable 


Parameter 

estimate 


Adjusted 

standard 

error 


t 


g 


Partial 

R-square 


Intercept 


3.5227 


0.2703 


8.41 


0.001 




WRPRATIO b 


4.5126 


0.4314 


6.75 


0.001 


0.1946 


Continuous enrollment 


-1.2149 


0.0854 


9.18 


0.001 


0.1287 


Community college transfer 


0.4789 


0.0632 


4.89 


0.001 


0.0616 


Four-to-four transfer 


0.3685 


0.0553 


4.30 


0.001 


0.0358 


Ever part-time 


0.3813 


0.0496 


4.96 


0.001 


0.0274 


Remedial problem 0 


0.1322 


0.0344 


2.48 


0.05 


0.0232 


Freshman GPA quintile 


0.0758 


0.0186 


2.63 


0.02 


0.0109 


Changed major 


0.1711 


0.0431 


2.56 


0.02 


0.0055 


SES quintile 


-0.0563 


0.0174 


2.09 


0.10 


0.0028 


Selective first institution 


-0.1227 


0.0470 


1.69 


t 


0.0045 


Academic Resources quintile 


-0.0551 


0.0233 


1.53 


t 


0.0052 


First-year low credits 


0.1717 


0.0800 


1.39 


t 


0.0015 


GPA trend 


-0.0518 


0.0288 


1.16 


t 


0.0024 


No delayed entry 


0.1922 


0.1190 


1.04 


t 


0.0014 


Co-ops or internships 


-0.0736 


0.0508 


0.87 


t 


0.0007 


Education anticipations 


-0.1324 


0.0497 


1.11 


t 


0.0006 


Gender 


0.0623 


0.0388 


1.04 


t 


0.0010 



R 2 = 0.5052 

Adjusted R 2 = 0.5056 

f Variable did not meet minimum criterion for statistical significance in this model. 
a Indicates the contribution of each variable to the explanation of variance, the total R 2 . 
b Ratio of no-penalty withdrawals and no-credit repeats to all courses attempted. 
c See Glossary, p. 191. 

NOTES: Root design effect = 1.55. Statistically significant variables are highlighted in bold. Weighted N=922k. 
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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And there is no question concerning which variables will contract time to degree: continuous 
enrollment and no remediation (the highest value of the "total remediation" variable). 
Socioeconomic status is a marginally significant contributor to lower time to degree, while the 
position of freshman year GPA quintile as a contributor to extended time to degree is somewhat 
puzzling. 

Of those variables that were admitted to the linear model but failed to reach the threshold of 
statistical significance, enrollment in cooperative education or internship courses deserves 
special attention because it raises contrasting policy options with respect to influences on time to 
degree. Knight (2004b) observes that: 

Timely degree completion is not all that matters in terms of college student outcomes. 
Both analytical and student self-report evidence supports the fact that enrollment in 
cooperative education classes, involvement in internships, etc., while extending time-to- 
degree, significantly improves student learning and skill development, affective 
outcomes, career prospects, and the like. Significantly reducing time-to-degree could 
perhaps demand a trade-off against other long-term (and maybe more important) 
outcomes, (p. 14). 

It turns out that the NELS:88/2000 baccalaureate students who engaged in cooperative education 
or internships or clinical externships did not take longer than others to complete degrees. But 
Knight is really asking readers to reflect on the principal objectives of conducting a 
baccalaureate enterprise. The modes of instructional delivery in professional and applied fields, 
in particular, often include carefully designed, graded experience in occupational environments, 
involving considerable time commitments that lie beyond the classroom. If we say to a school of 
engineering, “You can’t offer cooperative education any more and cannot require more than 128 
credits for a degree,” what do we accomplish in terms of the quality of student learning and the 
quality of engineering graduates? 

In contrasting spirit, if we observe institutions in which students can withdraw from courses, 
without penalty, as much as 10 weeks into a 14-week semester, a volume of withdrawals 
equivalent to 1 5 percent of all credits offered by the institution in a calendar year, and the 
average time to degree for native students at that institution to be well over five calendar years, 
we would accomplish a great deal by a tightening of course withdrawal policy. Here is a prime 
candidate for future research governed by quasi-experimental design: Find two comparable 
institutions (mission, size, demography, distribution of majors), one with lax withdrawal rules, 
the other with restrictive rules. The hypothesis, from everything learned in this data essay: An 
institution that restricts course withdrawal policy will witness higher graduation rates and shorter 
average time to degree. 
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APPENDIX L 



Tables and Comments on Miscellaneous Topics Raised in the Text 



Table LI. Percentage of 1988 eighth-graders who graduated from high school, by 

timing and type of high school diploma (if any), by gender, race/ethnicity, 
and socioeconomic status quintile 



Population 


Graduated 
by July 1992 
with standard 
diploma 


Graduated 
by July 1992 
with GED 


Graduated 
after July 1 992 
with standard 
diploma 


Graduated 

after 

July 1992 
with GED 


Had not 
graduated by 
December 2000 


All 1988 


eighth-graders 


78.3 (1.08) 


2.6(0.38) 


4.7 (0.44) 


5.1 (0.50) 


9.3 (0.84) 


Race/ethnicity 


White 


82.4(1.06) 


2.3 (0.40) 


3.1 (0.36) 


4.4 (0.54) 


7.8(0.85) 


African-American 


63.2 (4.59) 


4.6(1.86) 


11.2 (2.39) 


9.0 (2.22) 


12.0(3.27) 


Latino 


66.1 (3.34) 


2.5 (0.44) 


7.6(1.28) 


6.9(1.16) 


16.9(3.16) 


Asian 


93.4(1.90) 


0.4 (0.18) 


1.6 (0.57) 


0.9(0.35) 


3.7(1.81) 


American Indian 


61.6(7.71) 


2.9(1.24) 


11.3 (5.59) 


4.3 (2.39) 


20.0 (7.26) 


Gender 


Male 


77.1 (1.61) 


2.2 (0.41) 


5.1 (0.66) 


5.2 (0.80) 


10.4(1.37) 


Female 


79.6(1.28) 


2.9(0.63) 


4.2 (0.58) 


5.1 (0.58) 


8.3 (0.92) 


Socioeconomic 
status quintile 


Highest 


94.8 (1.12) 


0.5 (0.17) 


2.0 (0.58) 


1.6(0.75) 


1.1 (0.49) 


2nd quintile 


84.1 (1.90) 


3.7(1.30) 


4.4 (0.96) 


3.6(0.80) 


4.3 (1.02) 


3rd quintile 


83.4(1.99) 


2.4 (0.66) 


4.2 (1.01) 


5.3 (1.38) 


4.7 (0.73) 


4th quintile 


70.5 (2.55) 


2.9(0.80) 


5.9(1.00) 


7.7(1.34) 


13.0 (2.42) 


Lowest 


54.1 (2.61) 


3.6 (0.83) 


7.5 (1.25) 


8.0(1.15) 


26.8 (2.80) 


Urbanicity of 
high school 
community 


Urban 


73.7 (2.32) 


2.8 (0.75) 


7.1 (1.15) 


6.7(1.12) 


9.7(1.67) 


Suburban 


81.3 (1.41) 


2.6 (0.74) 


4.2 (0.62) 


4.3 (0.77) 


7.6 (0.92) 


Rural 


80.5 (1.85) 


2.3 (0.35) 


3.0 (0.46) 


4.6(0.711 


9.6(1.76) 



NOTES: Standard errors are in parentheses. Rows may not add to 100.0 percent due to rounding. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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Table L2. Percentage distribution of high school graduation status, postsecondary 

participation and degree completion of students in the 1992 NELS:88/2000 
survey, bv retention in-erade 




Was student ever retained in grade? 






No 


Yes 


Indeterminable 


Population 








All students: 


73.0 (0.94) 


17.8 (0.90) 


9.2 (0.71) 


Education Attainment 








High school completion 








Academic diploma 


90.6 (0.67) 


58.6 (2.44) 


57.1 (3.50) 


GED and other 


4.7 (0.43) 


20.2 (2.28) 


14.8 (2.31) 


Did not complete 


3.8(0.52) 


18.9(1.86) 


26.7 (3.60) 


Indeterminable 


0.9(0.18) 


2.2 (0.61) 


1.4 (0.37) 


Postsecondary entry 


81.3 (0.86) 


48.7(2.23) 


47.5 (3.53) 


Highest degree 








No postsecondary 


18.7(0.86) 


51.3 (2.23) 


52.5 (3.53) 


No degree 


35.1 (0.90) 


35.0(2.17) 


29.1 (2.93) 


Certificate 


3.7(0.32) 


3.7(0.65) 


4.1 (1.22) 


Associate 


6.2 (0.41) 


3.2 (0.61) 


4.8(1.33) 


Bachelor’s or 


36.4(1.00) 


6.8 (0.82) 


9.5 (1.91) 


above 








Highest degree of those 








who entered postsecondary 








No degree 


40.5 (1.01) 


69.0 (2.60) 


57.8 (5.19) 


Certificate 


4.8 (0.43) 


9.1 (1.59) 


9.8(3.23) 


Associate 


8.0(0.54) 


6.2 (0.95) 


10.7(2.86) 


Bachelor’s 


46.7(1.10) 


15.6(1.83) 


21.7(4.05) 



NOTES: (1) Standard errors are in parentheses. (2) Column totals for high school completion and highest degree 
may not add to 100.0 percent due to rounding. (3) Weighted Ns for all students: Never retained in grade = 2.298M; 
retained in grade = 560k; indeterminable = 291k. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES #2003- 
402 and Supplement). 
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Table L3. Percentage distribution of 1992 12th-graders who entered postsecondary 
education, by selectivity of first institution of attendance, by standard 
demographics 



Selectivity of first institution of attendance 



Population 


Highly 

selective 


Selective 


Non- 

selective 


Open- 

door 


Not 

ratable 3 


All students 


3.2 (0.43) 


12.2 (0.66) 


40.4(1.02) 


41.6(1.13) 


2.6(0.31) 


Gender 


Male 


3.4 (0.58) 


12.3 (0.86) 


38.2(1.40) 


44.0 (1.50) 


2.1 (0.39) 


Female 


3.0(0.51) 


12.1 (0.84) 


42.2 (1.30) 


39.6(1.44) 


3.1 (0.42) 


Race/ethnicity 


White 


2.4 (0.32) 


13.2 (0.78) 


42.5 (1.15) 


39.6 (1.24) 


2.3 (0.34) 


African-American 


5.1 (2.30) 


7.5 (1.59) 


41.7(3.36) 


42.4 (3.98) 


3.2 (0.82) 


Latino 


2.8(1.23) 


7.9(2.31) 


28.5 (3.09) 


56.2 (3.40) 


4.5 (1.47) 


Asian 


12.9(3.09) 


15.4 (2.35) 


30.6(3.11) 


38.4 (3.84) 


2.7(1.37) 


American Indian 


# 


6.7 (3.02) 


29.7(8.55) 


62.2 (9.33) 


1.4(1.07) 


Socioeconomic 
status quintile 

Highest 


7.3 (1.05) 


25.1 (1.47) 


45.6(1.67) 


20.6(1.60) 


1.3 (0.55) 


2nd quintile 


2.1 (0.85) 


10.2 (0.99) 


42.7(1.93) 


43.4 (2.07) 


1.6(0.33) 


3rd quintile 


0.4 (0.13) 


6.0 (0.72) 


40.2 (1.97) 


50.3 (2.10) 


3.0(0.52) 


4th quintile 


0.8 (0.29) 


4.8 (0.76) 


35.2 (1.97) 


55.7(2.06) 


3.5 (0.70) 


Lowest quintile 


1.3 (1.08) 


3.8(0.95) 


29.9 (3.07) 


58.9(3.36) 


6.1 (1.44) 



# Rounds to zero. 

a Includes less-than-two-year trade schools, conservatories (music and art), and schools of divinity. 

NOTES: Standard errors are in parentheses. Rows may not add to 100.0 percent due to rounding. 

Weighted N=2.03M. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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Table L4. Descriptive relationship of number of credits earned in the first calendar 
year of attendance by 1992 12th-graders to highest degree earned by 
December 2000 



Highest degree earned by December 2000 



Number of credits 








Bachelor’s 


Percentage of 


in first year 


None 


Certificate 


Associate 


or higher 


all students 


0-10 


84.6 (1.44) 


6.4 (0.98) 


5.4 (0.88) 


3.6 (0.75 


24.5 (0.94) 


11-19 


63.9 (2.27) 


4.9 (0.88) 


12.0(1.65) 


19.2 (1.70) 


16.5 (0.73) 


20-29 


27.5 (1.39) 


3.8(0.75) 


8.4 (0.79) 


60.3 (1.60) 


31.0 (0.83) 


30 or more 


10.6(0.86) 


5.1 (0.66) 


7.9(0.83) 


76.5 (1.31) 


27.9 (0.81) 


Among those who 
attended a four-year 
college at any time, 
number of credits 
earned in first year 












0-10 


75.8 (3.15) 


4.9(1.50) 


8.8 (2.18) 


10.4 (2.11) 


12.4 (0.88) 


11-19 


53.9 (2.82) 


2.1 (0.55) 


12.1 (2.22) 


31.9 (2.51) 


14.1 (0.70) 


20-29 


24.2 (1.39) 


2.0 (0.79) 


5.0 (0.62) 


68.9(1.59) 


38.8 (1.01) 


30 or more 


8.4 (0.81) 


0.6(0.16) 


3.4 (0.48) 


87.6 (0.93) 


34.8 (0.94) 



NOTES: Standard errors are in parentheses. Rows may not add to 100.0 percent due to rounding. Weighted N for 
all 1992 12th-graders = 2.09M; for those who attended a four-year college at any time:1.47M. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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Table L5. 



Percentage distribution of enrollment intensity over six years (1995/96-2001) 
of students who started out in postsecondary education in 1995/96, by age as 
of December 1995 



Age at entry Enrollment intensity 





Full-time 


Part-time 


Both full-time 




only 


only 


and Dart-time 


All Students 


47.0(1.1) 


12.2 (0.8) 


40.7(1.0) 


Less than 2 1 


51.8 (1.1) 


4.4 (0.5) 


43.8 (1.1) 


21-23 


40.0 (3.8) 


18.7 (3.1) 


41.3 (1.1) 


24-29 


30.2 (3.7) 


35.5 (4.6) 


34.3 (4.2) 


30 and older 


32.1 (3.1) 


42.9 (3.5) 


25.0 (2.8) 



NOTES: Standard errors are in parentheses. Rows may not add to 100.0 percent due to rounding. 

SOURCE: National Center for Education Statistics: Beginning Postsecondary Students Longitudinal Study, 
1995/96-2001, Data Analysis System (NCES 2003-171). 

As the reader will notice, about 48 percent of traditional-age students in the BPS longitudinal 
study of 1995/96-2001 told us that they attended part-time at some point in their undergraduate 
careers. In contrast, only 38 percent of the NELS:88/2000 postsecondary cohort indicated in 
their 2000 CATI interviews that they had attended part-time at any time. Are the two cohorts, 
three years apart in their modal starting date, that different? No. 

NELS:88/2000: 



Part-time by student report 


37.7 


Part-time by transcript evidence 


9.7 


Total part-time 


47.4 


BPS95/96-2001 students who started 
before they were 2 1 : 




Always part-time 


4.4 


Mixed full-time and part-time 


43.8 


Total part-time 


48.2 



So a dichotomous variable indicating whether the student ever attended part-time can be 
included in the logistic model in Step 5 (attendance patterns). 
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Table L6. Mean Carnegie unit credits earned in grades 10-12 by (a) all 1992 12th- 

graders and (b) by those of known socioeconomic status who attended a four- 
year college at any time and for whom high school transcript records and 
senior year test scores were complete 



Mean Carnegie units earned by: 



Category of 


All 1992 


All 1992 12th-graders 
who are the subjects 


earned credits 


12th-graders 


of The Toolbox Revisited 


All high school credits 


17.93 (0.09) 


18.74 (0.09) 


All academic credits 


16.06 (0.09) 


17.13 (0.09) 


All vocational credits 


1.28 (0.03) 


1.01 (0.03) 


English 


4.11 (0.02) 


4.18(0.02) 


Mathematics 


3.39(0.03) 


3.61 (0.03) 


Foreign language 


1.98 (0.04) 


2.31 (0.04) 


Science 


3.14(0.03) 


3.39(0.03) 


Social sciences 


3.55 (0.03) 


3.64 (0.03) 



a These students are the same as those described under (b) in the title of this table. 

NOTES: Standard errors are in parentheses. Weighted N for all 1992 12th-graders: 2.62M. Weighted N for 1992 
12th-graders who graduated from high school by December 1996, attended a four-year college at any time, and 
were of known socioeconomic status: 1.41M. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 



Table L7. Logistic account of the three components of Academic Resources in relation 
to bachelor’s degree attainment for all 1992 12th-graders with complete 
records for all three components 







Adjusted 








Standard- 






Parameter 


standard 








ized 




Variable 


estimate 


error 


F 


t 


JL 


estimate 


Odds Ratio 


Intercept 


-5.3515 


0.2348 


529.46 11.12 


0.01 






Curriculum intensity 


0.6273 


0.0568 


122.17 


5.39 


0.02 


0.4672 


1.87 


Classrank/GPA 


0.4700 


0.0529 


79.05 


4.34 


0.05 


0.3686 


1.60 


Senior test score 


0.3649 


0.0561 


42.25 


3.17 


0.10 


0.2785 


1.44 



NOTES: Standard errors adjusted by root design effect=2.05; universe consists of 1992 12th-graders with high 
school transcript records and senior year test scores. Percent of concordant probabilities predicted: 85.1; 
G 2 =7536.7; df= 8280; G 2 /<i/^90.9; X 2 (df)=2977.32 (3). 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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Table L8a. First season of postsecondary attendance of NELS: 88/2000 students 
(a) who participated in the 1992 NELS survey and (b) who were 





All students in 
1992 survey 


12th-graders in 1992 
with full high school 
records who attended 
a four-year college 


First season of attendance 


Summer term 


5.8 (0.46) 


4.4 (0.49) 


Fall term 


82.1 (0.83) 


91.3 (0.65) 


Winter or spring tenns 


12.1 (0.71) 


4.3 (0.37) 



Table L8b. Percentage of two groups of NELS:88/2000 postsecondary students who 
entered in the fall term: (a) All who participated in the 1992 NELS 
survey and (b) those who were 12th-graders in 1992 and subsequently 
attended a four-year college, by race/ethnicity and socioeconomic status 
quintile 



Percent Entering in the Fall Term 

12th-graders in 1992 
with full high school 
All students in records who attended 

1992 survey a four-year college 



By race/ethnicity 



White 


84.3 (0.84) 


92.3 (0.54) 


African-American 


73.8 (3.53) 


85.7 (4.28) 


Latino 


73.9(2.19) 


86.1 (3.49) 


Asian 


83.3 (3.54) 


89.9 (3.77) 


American Indian 


85.1 (5.82) 


97.1 (2.87) 


By socioeconomic status 
quintile 






Highest 


92.2 (0.73) 


93.7(0.81) 


2nd quintile 


83.3 (1.85) 


90.6(1.66) 


3rd quintile 


79.6(1.68) 


90.8 (1.04) 


4th quintile 


77.4(1.83) 


89.9(1.32) 


Lowest 


66.0 (2.96) 


82.7 (3.62) 



NOTES: Standard errors are in parentheses. Weighted Ns: 1992 survey participants = 2.18M; 1992 12th-graders 
who attended a four-year college at any time and who presented full high school records = 1.1 8M. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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Table L9. Percentage of 1992 12th-graders earning credits in college-level math courses, 
both in the first calendar year of attendance and cumulatively, by number of 
credits earned in college-level mathematics, and type and selectivity of first 
institution of attendance 



Number of credits in all college-level 
math courses earned in 
the first calendar year of attendance 

Characteristics of 



first institution 


None 


M 


More than 4 


Type of first institution 








Four-year 
Community college 
Other sub-baccalaureate 


44.2(1.20) 
82.1 (1.22) 
89.2 (2.72) 


30.2(1.02) 
12.0(1.00) 
9.5 (2.70) 


25.6 (0.98) 
6.0(0.75) 
1.3 (0.51) 


Selectivity of first institution 








Highly selective 
Selective 
Nonselective 
Open-door 
Not ratable 


25.1 (5.69) 
26.9(1.98) 

51.3 (1.26) 
82.4(1.17) 

92.4 (2.51) 


26.9 (4.24) 
32.0(2.38) 
30.1 (1.15) 
11.8(0.96) 
3.7 (2.09) 


48.0(5.32) 
41.1 (2,28) 
18.6 (0.91) 
5.8(0.73) 
3.9(1.42) 



Number of credits in all college-level math courses earned at 
any time in the undergraduate careers of students who earned 
more than 10 undergraduate credits 2 



Type of first institution 


None 


1-4 


5-9 


More than 9 


Four-year 
Community college 
Other sub-baccalaureate 


23.2 (0.96) 
52.8 (1.82) 
79.1 (3.88) 


26.3 (0.96) 
21.4(1.47) 

12.4 (3.62) 


28.9 (0.92) 
16.1 (1.29) 
6.1 (1.70) 


21.5 (0.99) 
9.6(1.01) 
2.4 (0.83) 


Selectivity of first institution 










Highly selective 
Selective 
Nonselective 
Open-door 
Not ratable 


18.1 (5.44) 
16.5 (1.74) 
25.4(1.02) 
54.2 (1.77) 
88.0(3.101 


17.8(3.53) 
21.1 (2.21) 
29.1 (1.14) 
21.2(1.43) 
2.7(1.14) 


18.0(3.34) 

31.9(2.13) 

28.7(1.09) 

15.6(1.24) 

7.9(2.65) 


46.1 (5.33) 
20.5 (2.16) 
16.8 (0.94) 
9.1 (0.92) 
1.4(0.81) 



The "more than 10 undergraduate credits" criterion excludes incidental students. 



NOTES: Standard errors are in parentheses. Rows may not add to 100.0 percent due to rounding. Weighted N for 
all 1992 12th-graders = 2.09M; for those who earned more than ten credits: 1.83M. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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Table L10. Logistic account of factors associated with bachelor’s degree completion 
for 1992 12th-graders who attended a four- year college at any time, using 
second-year cumulative credits and second-year cumulative GPA 







Adjusted 










Parameter 


standard 








Variable 


estimate 


error 


t 


J2- 


Delta-n 


Intercept 


-6.2118 


.7595 


4.63 


.001 


— 


GPA through second year 


0.7585 


.1826 


2.37 


.05 


0.1672 


Credits through second year 


1.1780 


.1054 


6.34 


.001 


0.2597 


No delay of entry 


0.8852 


.3240 


1.55 


t 


t 


SES quintile 


0.2636 


.0651 


2.29 


.05 


0.0581 


Education expectations 


0.5402 


.2031 


1.51 


t 


t 


Academic Resources quintile 


0.3345 


.0821 


2.31 


.05 


0.0737 


Ever part-time 


-1.3259 


.1611 


4.66 


.001 


-0.2923 


Summer-term credits 


0.5273 


.0929 


3.29 


.01 


0.1162 


Race/ethnicity 


-0.3327 


.2186 


0.86 


t 


t 


Gender 


-0.1998 


.1574 


0.72 


t 


t 


Parenthood by age 20 


-0.8960 


.4535 


1.12 


t 


t 


Multiple institutions 


-0.5354 


.1874 


1.62 


t 


t 


Community college transfer 


0.6251 


.2151 


1.65 


t 


t 


First institution was selective 


0.4008 


.2248 


1.01 


t 


t 


College-level math in first year 


0.2064 


.1692 


0.69 


t 


t 


First-year remediation 


0.2940 


.1929 


0.86 





1 



t Variables did not meet threshold criterion for statistical significance. 

NOTES: Standard errors adjusted by root design effect = 1.71. Delta-p computed only for statistically significant 
variables (in bold). Universe consists of all 1992 12th-graders who participated in postsecondary education, whose 
records indicated positive values for all variables in the model, and whose transcript records were complete. 
Weighted N = 1.5M. G 2 = 3477.1; t//=4759; G 2 /df= 0.73; X 2 (df )= 2272.2 (17); p = 0.0001. Proportion concordant 
predicted probabilities = 90.4 percent. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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Table Lll. Institutional attendance patterns of 1982 and 1992 12th-graders who 
subsequently earned more than 10 postsecondary credits 3 



Attendance characteristics 


Class of 1982 


Class of 1992 


Number of schools attended 


One 


46.9 (0.82) 


43.5 (1.06) 


Two 


33.8 (0.75) 


35.1 (0.93) 


More than two 


19.3 (0.63) 


21.5 (0.84) 


For those who attended more 
than one school: 


Attended in more than one state 


39.5 (1.21) 


35.7(1.36) 


Institutional combinations: 


Four-year only 


38.9(1.21) 


27.5 (1.17) 


Reverse transfer 15 


7.6 (0.60) 


8.2 (0.76) 


Community college transfer 


18.1 (0.90) 


22.1 (1.12) 


Alternating/ simultaneous 0 


10.2 (0.67) 


13.8 (1.01) 


Four-year student with 




12.7(0.90) 


incidental community college d 


e 




Four-year plus other sub- 


4.9 (0.46) 


1.8(0.36) 


baccalaureate 


Community college only 


10.2 (0.69) 


9.1 (0.71) 


Community college and other 


5.3 (0.50) 


3.0(0.33) 


sub-baccalaureate 


Sub-baccalaureate trade school 


1.1 (0.22) 


0.3 (0.20) 


only 


Other combinations 


3.6(0.51) 


1.5 (0.31) 



a The reason for excluding incidental students (those who earned 10 or fewer credits) is that they provide an 
insufficient history forjudging multi-institutional attendance. 

b Student began in four-year institution, did not earn a degree, and transferred to a community college. 
c Student moved back and forth between four-year and two-year institutions, or attended two institutions at the same 
time, or did both. 

d Student was based in a four-year institution, and earned 10 or fewer credits, principally during summer terms, from 
community colleges. This measure was not available in the High School & Beyond/Sophomore cohort transcript 
file. 

e Category was not available in the High School & Beyond/Sophomore cohort transcript files (NCES 2000-194). 
NOTES: Standard errors are in parentheses. Rows may not add to 100.0 percent due to rounding. Weighted N for 
those who earned more than ten credits: 1.83M. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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Table L12. Percentage distribution of final (December 2000) education status of 1988 
eighth-graders, by type and timing of high school diploma (if any), and 
including those who did not earn high school diplomas 

Secondary/postsecondary status Percent 

1) Earned standard high school diploma by July 1992 and 

Earned at least a bachelor’s degree 
Associate degree was highest earned credential 
Certificate was highest earned credential 
No degree, but still enrolled in 2000 
No degree, not enrolled in 2000 
Never entered postsecondary education 

2) Earned standard high school diploma after July 1992 and 

Earned at least a bachelor’s degree 
Associate degree was highest earned credential 
Certificate was highest earned credential 
No degree, but still enrolled in 2000 
No degree, not enrolled in 2000 
Never entered postsecondary education 

3) Earned GEDs or certificates of attendance and 

Earned at least a bachelor’s degree 
Associate degree was highest earned credential 
Certificate was highest earned credential 
No degree, but still enrolled in 2000 
No degree, not enrolled in 2000 
Never entered postsecondary education 

4) Others 

Did not graduate from high school, but entered postsecondary 1 .0 (0.42) 

Did not graduate from high school, no postsecondary 6.7 (0.65) 

Indeterminable high school graduation status 1.7 (0.37) 

NOTES: Standard errors are in parentheses. Percent column may not add to 100.0 due to rounding. Weight used 
throughout this table is the F4BYWT with a base year (1988) flag. F4BYWT covers NELS:88/2000 students who 
were in both the base year (1988) sample and the 2000 follow-up survey sample. Weighted N=2.93M. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 



0.2 ( 0 . 12 ) 
0.3 (0.14) 
0.2 (0.04) 
0.4 (0.14) 
1.9(0.29) 
1.6(0.23) 



0.1 (0.03) 
0.2 (0.05) 
0.4 (0.14) 
0.8 ( 0 . 22 ) 
2.8(0.38) 
3.9 (0.46) 



29.1 (0.94) 

4.7 (0.28) 

2.8 (0.24) 
4.6(0.33) 

23.6 (0.79) 

13.1 (1.10) 
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Table L13. Mean elapsed time to bachelor’s degree 3 for all 1992 12th-graders who 
earned the degree by December 2000, by demographics, academic 
background, and attendance pattern characteristics 

Mean elapsed 
calendar 

years Standard Standard 

Student characteristic to degree deviation error 



All NELS:88/2000 bachelor’s 


4.58 


1.098 


0.028 


By Academic Resources quintile 


Highest quintile 


4.23 


0.914 


0.042 


2nd quintile 


4.63 


1.089 


0.045 


3rd quintile 


4.94 


1.091 


0.071 


4th quintile 


4.98 


1.206 


0.137 


Lowest quintile 


5.54 


1.095 


0.161 


By socioeconomic status quintile 


Highest quintile 


4.40 


1.018 


0.044 


2nd quintile 


4.66 


1.131 


0.056 


3rd quintile 


4.74 


1.089 


0.060 


4th quintile 


4.83 


1.201 


0.084 


Lowest quintile 


4.87 


1.113 


0.120 


By education anticipations 


Bachelor’s consistent 


4.49 


1.051 


0.029 


Raised to bachelor’s 


4.90 


1.138 


0.067 


Lowered from bachelor’s 


5.14 


1.303 


0.218 


By selectivity of first institution 


Highly selective 


3.92 


0.741 


0.093 


Selective 


4.32 


0.987 


0.071 


Nonselective 


4.53 


1.026 


0.031 


Open-door 


5.39 


1.165 


0.070 


Not rated 


4.85 


1.412 


0.481 


By number of acceleration credits 


None 


4.67 


1.105 


0.034 


1-4 


4.46 


1.006 


0.083 


5-8 


4.38 


1.005 


0.081 


9 or more 


4.28 


1.101 


0.088 



See notes at end of table. 
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Table L13. Mean elapsed time to bachelor’s degree 3 for all 1992 12th-graders who 
earned the degree by December 2000, by demographics, academic 
background, and attendance pattern characteristics — continued 


Student characteristic 


Mean elapsed 
calendar 
years 
to degree 


Standard 

deviation 


Standard 

error 


By number of schools attended 


One 


4.30 


0.825 


0.034 


Two 


4.66 


1.143 


0.048 


Three or more 


5.04 


1.254 


0.063 


By transfer status 


Community college to four-year 


5.43 


1.162 


0.071 


Four-year to four-year 


5.09 


1.197 


0.060 


No transfer 


4.43 


1.015 


0.030 


By continuity of enrollment 


Continuously enrolled 


4.44 


0.941 


0.024 


Noncontinuous 


6.68 


1.114 


0.101 


By number of remedial courses 


None 


4.43 


1.042 


0.033 


One 


4.94 


1.117 


0.071 


More than one 


5.33 


1.092 


0.068 


Bachelor’s degree major 


Business 


4.56 


1.083 


0.072 


Education 


4.72 


1.193 


0.088 


Engineering/ architecture 


4.71 


1.019 


0.089 


Math/computer science 


4.45 


0.915 


0.161 


Physical sciences 


4.99 


1.293 


0.194 


Biological sciences 


4.41 


0.998 


0.063 


Health sciences/services 


4.66 


1.131 


0.081 


Humanities 


4.35 


1.134 


0.143 


Fine and performing arts 


4.61 


1.183 


0.144 


Social sciences 


4.50 


1.062 


0.058 


Annlied social sciences’ 5 


4.68 


1.033 


0.066 



“From first true date of attendance to degree award date. 



b Includes communications, public administration, criminal justice, social work, family and community services, and 
leisure studies and recreation. 

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement). 
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GLOSSARY 



VARIABLES AND THEIR CONSTRUCTION 

This glossary serves not only to expand the definition of the variables considered for The Tool- 
box Revisited, but also to provide details on their construction and allied guidance for 
interpretation that the reader may find helpful. As in Part II of this study, the presentation 
follows the order of the seven steps of the logistic narrative. 

Where the initial definition of the variables in Part II was simple and transparent, e.g., GENDER, 
no further elaboration is provided here. Nearly all these variables were either taken directly or 
derived from other variables on NCES 2003-402 and its June 2004 Supplement. 

Step la: Demographic Background 

NNSE (Nonnative speaker of English) — New in this analysis. 

The most notable demographic change between postsecondary students from the High School 
Class of 1982 and those from the High School Class of 1992 was a doubling of those who not 
only came from households where English was a second language, but who themselves were 
users of the heritage language. More than half of both Asian and Latino postsecondary students 
were so classified (Adehnan 2004a, table 1.3, p. 9). 

IMMIG (Family immigrant status) — New in this analysis. 

Allied to second language background is the immigrant status of families. A dichotomous 
variable, IMMIG, was created to indicate whether the student’s parents were immigrants to the 
United States within the previous 10 years. Among those who attended a four-year college at 
any time, 27.5 percent (s.e. = 3.33) of Asian students and 8.8 percent (s.e. = 1.61) of Latino 
students came from recent immigrant families. The dichotomous variable did not meet the 
minimum t criterion for retention in the demographic logistic model. Nonresponse bias 
undermines this variable: Responses from nearly 15 percent of the NELS:88/2000 families are 
missing. 

BROSIS (Number of siblings) — New in this analysis. 

Some research has indicated that students from larger families have more difficulty entering and 
successfully participating in higher education (Stage and Hossler 1989; Leigh and Gil 2003; 
Perkhounkova, McLaughlin, and Noble 2004). The dichotomous variable, BROSIS, marks 
students with three or more siblings versus those with one, two, or none. 

FIRSTGEN (First generation postsecondary student) — New in this analysis. 

A dichotomous variable under which the criterion for first-generation status is that neither parent 
had attended a postsecondary institution. Parents’ education status was reported by parents 
themselves, not by students. The discrepancy between students’ understandings of their parents’ 
education status and parental accounts was demonstrated in the original Tool Box (p. 36): For the 
High School Class of 1982, only 50 percent of the students with at least one parent who had 
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earned a bachelor’s degree (but not a graduate degree) could correctly describe their parents’ 
achievement. Parents’ highest level of education is a key component of socioeconomic status, 
and the original Tool Box did not test it separately. 

FAMINC (Family income) — New in this analysis. 

Family income in 1991 (the year before NELS students were scheduled to be in the 12th grade) 
was reported by parents, and set in six bands with a fairly even distribution of all respondents 
across those bands. In 2005 dollars (adjusted for a Gross Domestic Product deflator 66 of 1.288), 
as shown in figure 4, these bands were: 

Figure 4. 1991 family income bands used in the NELS:88/2000 in 2005 dollars 



Income band Income range ($000) 

Highest 96.6 and higher 

Second 64.4-96.5 

Third 45.1-64.3 

Fourth 32.2-45.0 

Fifth 19.3-32.1 

Lowest Less than 19.3 

SOURCES: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003- 
402 and Supplement); Budget of the United States Government: Historical Tables, Fiscal Year 2005, table 10.1. 



For purposes of our analysis, the bands were combined in pairs — highest two, middle two, and 
lowest two — to produce a trichotomy. This version is used in the Step la logistic model 
(demographic characteristics only), along with FIRSTGEN, as one of the proxies for 
socioeconomic status. 

NEWCHILD (Parenthood by age 20) — Carried forward from Answers in the Tool Box. 

The variable, abbreviated as NEWCHILD, is a dichotomous marking of students (of either sex) 
who became parents by age 20. It was a powerful, negative, and consistently significant variable 
through all stages of the logistic story of the HS&B/So. A decade later, the rate of early 
parenthood had changed. For the entire cohort of the HS&B/So, 1 1.6 percent (s.e. = 0.43) 
became parents by age 20, a proportion that rose to 16.4 percent (s.e. = 0.92) for the 
NELS:88/2000. For those who ever entered a postsecondary institution, these percentages drop 
in half. Georges (2000) would remind us of the common sense that students who became parents 
by age 20 were not likely to enter college at all by that time. Indeed, 59.4 percent (s.e. = 2.30) of 



66 For the GDP deflator calculator, go to http://wwwl.jsc.nasa.gov/bu2/inflateGDP.html 
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the NELS:88/2000 participants who had children by 1994 did not enter any postsecondary 
institution by 2000, compared with 20.9 percent (s.e. = 0.89) of those who did not become 
parents by 1994. 67 

SESQUINT (Socioeconomic status quintile) — Carried forward from the original Tool Box. 

In NCES longitudinal studies, socioeconomic status (abbreviated SES) is a composite index of 
family income, parents’ highest level of education, prestige of parents’ occupations, and the 
presence of items such as books, regular newspapers, and a computer in the student’s household. 
Parents provide this infonnation, and the 1992 parent interview was the primary source. The 
NELS:88/2000 files presented this index in centiles, as did the HS& Beyond/So file before it. 

To match quintile presentations of high school curriculum intensity, high school class 
rank/Grade Point Average, 12th-grade test score, and the composite variable, Academic 
Resources (see below), the centile scale was collapsed to quintiles. SES is an anchor of the 
education attainment literature, with recent studies enlightening our understanding of the 
complex ways students of different SES backgrounds experience higher education (e.g., Walpole 
2003). 



Step lb: High School Background 

EDUANTIC (Education anticipations) — Modified from Answers in the Tool Box. 

In both the original Tool Box and The Toolbox Revisited, this is a composite variable describing 
the consistency and level of the student’s education expectations, hence, goal commitment 
(Allen and Nora 1995). It departs considerably from measures of student education 
“aspirations” that are based on the student’s answer to a single question at one moment in time. 
The variable was built from matched pairs of questions asked in grades 10 and 12. The High 
School & Beyond/Sophomore cohort surveys included six pairs of questions: highest level of 
education expected, planned dominant activity in the year following high school graduation, 
timing of college entry, choice of a two-year or four-year college as entry point, lowest level of 
education with which the student would be satisfied, and whether the student would be 
disappointed if he or she did not graduate from college. 

The NELS:88/2000 version does not have as strong a base. Only two pairs of questions were 
asked in grades 10 and 12 that feed into the construction: highest level of education expected and 
timing of college entry. 68 In grade 12 only, the respondent was asked what was the most likely 
type of postsecondary institution he or she would enter: four-year, two-year, or trade school, and 
the answer to this question also was used in building the variable. But the lack of parallelism 



67 These percentages are based on the entire panel of participants in 1992 who also participated in the 1994 
and 2000 follow-up surveys. 

68 Georges (2000) used a similar level-of-expectations by consistency-of-expectations construct for parental 
visions of their children’s future education in the NELS:88/2000 cohort. Parents were interviewed on this issue 
when students were in the eighth grade and again when students were scheduled to be in the 12th grade. 
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between the HS&B/So survey and the NELS:88/2000 lead to revisiting the NELS categories and 
applying stricter algorithmic standards to yield an education anticipations variable closer to the 
structure of its predecessor, as displayed in figure 5. 

Even after this construction, the education anticipations variable required an adjustment. Unlike 
the quintile variables for curriculum, class rank/GPA, and senior year test score, in which 
meanings are linear, the five-level anticipations variable is categorical. For purposes of 
multivariate analysis, the five levels were reduced to three as follows: (1) consistently expected 
to earn a bachelor’s degree; (2) raised expectations to the bachelor’s degree between grades 10 
and 12; and (3) either lowered expectations from the bachelor’s level between grades 10 and 12 
or never expected to earn a bachelor’s degree. The result is still a categorical variable, but the 
reference point — expectations for earning a bachelor’s degree — is now constant in all three 
values. 

Figure 5. Comparison of the education anticipations composite variable in the High 
School & Beyond/Sophomore cohort and the NELS: 88/2000: Changes in 
students’ expectations between grade 10 and grade 12 



HS& B/So 

Bachelor’s consistent 
Raised to bachelor’s 



NELS:88/2000 
Bachelor’s consistent 
Raised to bachelor’s 



Lowered from bachelor’s to some college; 
associate consistent 



Lowered from bachelor’s to some 
college, inconsistent 



Lowered to no degree; 
associate inconsistent 



Raised to some college; consistent 
“some college” 



No degree plans, don’t know Sub-baccalaureate, including high 

school diploma or less 

SOURCES: National Center for Education Statistics: High School & Beyond/Sophomore cohort, NCES 2000-194; 
NELS:88/2000 Postsecondary Transcript Files, NCES 2003-402. 

CLSSRNKQ (Class rank/GPA quintile) — Carried forward from the original Tool Box. 

This variable is a composite of high school class rank and GPA quintile. In both the HS&B/So 
and NELS:88/2000 data sets the composite was constructed in roughly the same way. The 
construction is described in detail in Appendix C, but the key features are worth repeating here. 

The construction of this variable began with class rank, expressed as a percentile, for students 
whose high school graduating classes were larger than 10. Class rank was chosen for the base 
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reference because it overrides variability in local grading practices, 69 and responds to the 
divergence between stable scores on standardized tests such as the SAT or ACT and rising GPAs 
among the college bound (College Board 1998). Not all high schools compute class rank, 70 so 
for a significant percentage of students, this datum is missing. For these missing cases (as well 
as for the students from very small high schools), the variable construction turns to high school 
GPA (where available, and only for students with three or more years of course work). Basing a 
percentage scale first on class rank, and filling in the missing values with available GPAs from a 
matched scale using an equipercentile concordance methodology (Houston and Sawyer 1991), 
weighting the combined scale and cutting it by quintiles solves some of the bias and validity 
problems that would result from relying on GPA alone. 

SRTSQUIN (Senior year test score quintile) — Carried forward from the original Tool Box. 

SRTSQUIN is the quintile version of a senior year test score. Comparing the HS&B/So to the 
NELS:88/2000, this variable had the least variation in construction among the precollegiate 
academic performance variables. In both data sets, composite scores on an “enhanced mini- 
SAT” given to survey participants in the 12th grade were set out in percentiles. For missing 
cases, ACT scores were first converted to the SAT scale using ACT’s equipercentile 
concordance methodology (accounting for the 1989 revision of the ACT test battery), and a 
combined SAT/ACT variable constructed. Scores on this variable were set out in percentiles, 
and filled in for missing cases of the senior year test score percentile using the equipercentile 
concordance method. SRTSQUIN reduces the noise in the lumpy distribution with a quintile 
presentation. Despite these efforts, 14 percent of the students in the NELS:88/2000 database are 
missing a senior year test measure, compared to 9 percent in the HS&B/So (see Appendix C). It 
must be acknowledged that the construction of SRTSQUIN combines low-stakes test scores with 
high-stakes test scores, though it is unclear how that ultimately affects a quintile presentation. 

HIGHMATH (Highest level of mathematics reached in high school) — Carried forward from 
the original Tool Box. 

In the original construction of this variable in both the HS&B/So and NELS:88/2000 data sets, 
some editing of the original coding of high school mathematics courses was required in light of 
postsecondary transcript evidence. For example, if the original coding in 1992 judged the 
student’s highest level of high school mathematics to be "geometry," and the student’s 
postsecondary transcript record of 2000 showed enrollment and completion of "Calculus III" in 
the first semester of college, then it was obvious that the original coder did not know the 



69 Among the divergent policies found by the College Board (1998): (a) 9 percent of high schools do not 
use A-F or numeric grading systems; (b) 8 percent of high schools do not give credit for grades lower than C-; (c) 
19 percent of high schools are in districts where grading systems vary; and (d) 43 percent of high schools exclude 
some courses when calculating GPA. 

7l) ln its survey of high school grading practices, the College Board (1998) found that 19 percent of high 
schools did not calculate class rank. 
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difference between "geometry" and "analytic geometry," and the student’s high school record 
had to be adjusted accordingly. HIGHMATH is considered a categorical variable (as opposed to 
a continuous variable) because the distance between its levels cannot be judged as ordinal. 

SCIMOM (High school momentum in science and mathematics) — New in this analysis. 

This variable has three values: (1) A student who reached a level of math beyond Algebra 2 and 
who earned three or more Carnegie units in core laboratory science was labeled as having 
sufficient momentum in STEM (Science, Technology, Engineering, and Mathematics)-oriented 
fields to enhance that momentum even further in a postsecondary environment (whether or not 
they chose to do so). (2) Students who reached Algebra 2 but no higher, and accumulated more 
than 1.5 units of core laboratory science exhibited "modest" science momentum, and (3) those 
whose high school records fell short of "modest" on either the mathematics or science criteria 
were labeled "weak-to-minimal" in terms of science momentum. SCIMOM is one of three 
variables tested together as proxies for the combined measure of the academic intensity of a 
student’s high school curriculum, HSCURRQ. 

FLAN (Number of units of foreign language in high school) — New in this analysis. 

This variable was developed as part of a configuration of proxies for academic curriculum 
intensity so that we could test the proxies against the consolidated academic curriculum index, 
HSCURRQ, described on pages 24-27 and in Appendix F. FLAN indicates the number of 
Carnegie units earned in high school foreign language study, and has five values: more than 3 
units, from 2.01 to 3 units, from 1.01 to 2 units, from 0.1 to 1 unit, and 0 units. FLAN is one of 
three variables tested as proxies for HSCURRQ. 

ADVANCE (Number of Advanced Placement courses) — New in this analysis. 

This variable was constructed with three values based simply on the number of AP courses 
recorded: three or more, one or two, and none. It was tested, along with SCIMOM and FLAN, 
as a proxy for the consolidated academic curriculum index, HSCURRQ. 

The identification of Advanced Placement course work in the NELS:88/2000 was a complex 
task. The version of the NELS:88/2000 high school transcripts to which researchers have referred 
in the past understates participation in AP courses, though it was difficult to say precisely by how 
much. Two proxy measures were employed to create the first version of an AP variable on the 
data file used in this study: 71 AP test scores recorded on the high school transcripts and AP 
entries on the postsecondary transcripts (which do not include test scores). These were combined 
to yield 7 percent of the NELS:88/2000 12th-graders, compared with the College Board’s 



71 NCES CD 2003-402. 
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estimate of approximately 1 1 percent of 1992 12th-graders having taken AP courses. 72 Not 
everyone who takes AP courses takes AP examinations, and while that might explain some of 
the gap in the estimates, it still left an uncomfortable discrepancy between the NELS and College 
Board accounts of Advanced Placement volume. 

In light of this low yield and anomalies in the data, a new approach was taken in the work that 
produced the 2004 Supplement to the original NCES 2003-402 fdes. Enough new cases of 
AP course taking {not courses labeled "honors" or just "advanced") were identified to raise the 
proportion of NELS 12th-graders with AP courses to 9.4 percent. 72 

Step 2: Postsecondary Entry 

FIRST4 (First school attended was a four- year college) — Carried forward from Answers in 
the Tool Box. 

The dichotomous variable indicating whether the first institution attended by the student was a 
four-year school was active in the original Tool Box account, but did not qualify for entry into 
the model in this replication. It is not clear why this should be the case. Some 81.9 percent 
(s.e. = 0.82) of the class of 1982 who attended a four-year college at any time started at a four- 
year college, versus 78.1 percent (s.e. = 1.04) of the class of 1992. There is no meaningful 
difference in those numbers. 

DOCT (First school attended was a doctoral degree-granting institution) — Carried forward 
from the original Tool Box. 

Another dichotomous variable indicating that the first institution attended by the student was not 
only a four-year college, but a doctoral degree-granting school. In the original Tool Box, this 
variable was admitted to the steps of the logistic model but was never statistically significant. In 
The Toolbox Revisited analysis, this variable did not qualify for entrance in the model, even 
though roughly similar percentages of students who attended a four-year college at any time 
started in a doctoral degree-granting institution (33.8 percent; s.e. = 1.06 for the HS&B/So 
versus 34.1 percent; s.e. = 1.11 for the NELS:88/2000). 
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“See indicator #14 in The Condition of Education, 1999. Washington, D.C.: National Center for 
Education Statistics, 1999, p. 56. 

73 

This approach began by isolating students who were consistent in their claims to have taken an 
"advanced placement course," planned to take an AP test, and reported taking at least one AP test between 1990 and 
1993. From this group, those for whom AP course work had already been recorded were dropped. The balance 
were first matched against the HIGHMATH variable. For students who reached calculus or precalculus, responses 
to another question as to whether they took the course to earn college credit were invoked. If the student responded 
positively, the record was credited with an AP course. College transcripts were also examined for foreign language 
study at advanced levels in the first year of attendance (along with more than 2 years of foreign language study in 
high school), etc. Where matches were found, the student record was credited with an AP course. 
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The variable indicating the selectivity of the first institution of attendance (SELECT) is, to a 
large extent, confounded with doctoral degree-granting status (55 percent of the NELS:88/2000 
students who started at doctoral degree-granting institutions started at either selective or highly 
selective schools). 

SELECT (Selectivity of first institution attended) — Carried forward from the original Tool 
Box. 

A dichotomous variable indicating that the first institution attended by the student was either 
highly selective or selective. 74 Some 4. 1 percent (s.e. 0.49) of the universe for this study started 
in highly selective institutions (e.g., Princeton, the University of California at Berkeley, and 
Harvey Mudd). Another 17.8 percent (s.e. = 0.98) began their postsecondary careers in selective 
institutions (principally flagship state universities such as Michigan-Ann Arbor, North Carolina- 
Chapel Hill, Wisconsin-Madison, and Texas- Austin). More than half of the universe of students 
in this study (56.3 percent; s.e. = 1.32) commenced study in nonselective four-year colleges, and 
20.7 percent (s.e. = 1.19) in open-door institutions, a category that is just about identical to 
community colleges. For the previous cohort of students from the High School Class of 1982, 
selectivity of first institution had a modestly positive and statistically significant association with 
degree completion until the last step of that analysis, extended postsecondary perfonnance. 
Readers interested in the distribution of first institutional selectivity by race/ethnicity, gender, 
and socioeconomic status are referred to Appendix L, table L3. 

NODELAY (Direct entry to postsecondary education following high school) — Carried 
forward from the original Tool Box. 

This dichotomous variable was built from a simple distinction: Among students for whom we 
know both the month and year of high school graduation and the month and year of a transcript- 
documented attendance in a postsecondary institution following high school graduation, is the 
difference (a) seven months or less, (b) 8-18 months, or (c) more than 18 months? The first of 
these options is regarded as "direct entry." Dual enrollment courses do not count in this 
determination. The student who graduates from high school in June has until the following 
January to enroll and be judged "no delay." To those who might question the seven month 
allowance and argue for four months, one would point to a small percentage of graduates with 
April commencement dates who would be judged late entrants in September under a four-month 
rule. 



74 Institutional selectivity in the postsecondary files of all three grade-cohort longitudinal studies 
conducted by the National Center for Education Statistics has five values: highly selective, selective, nonselective, 
open-door, and not ratable. The first three of these values were based on the selectivity cells developed by the 
Cooperative Institutional Research Project (CIRP) at UCLA for its annual survey (since 1966) of entering freshmen. 
Community colleges and area vocational-technical institutes (AVTIs) were assigned the value of “open-door.” 
Theological seminaries, music conservatories, and sub-baccalaureate vocational schools were considered “not 
ratable.” 
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ACCELCRD (Credits earned prior to high school graduation and by examination) — New 

in this analysis. 

ACCELCRD sums all college credits earned by course work prior to high school graduation, 
along with credits earned by examination — including Advanced Placement, College Level 
Examination Program (CLEP), and institutional challenge exams (the majority of these — in the 
records of the NELS:88/2000 — in foreign languages). Most of these credits were earned either 
prior to matriculation or during the first term of enrollment, though some were earned at later 
points in the student’s undergraduate career. The ACCELCRD values were set at three levels: 
more than 4 credits, 1-4, and zero. 

Step 3: First-year Performance 

LOWCRED (Less than 20 additive credits earned in the first calendar year of 
attendance) — Carried forward from Answers in the Tool Box. 

A dichotomous variable built on an analysis of the mean number of additive credits (those that 
count toward degrees) earned by all postsecondary students in the first calendar year following 
the date of first enrollment. The threshold at which this broad population began to earn any 
credentials (including certificates and associate degrees) was 20 credits (in the NELS:88/2000 
population, 20.252 credits, to be precise; s.e. = 1.016). LOWCRED is an early momentum 
indicator with enduring consequences. As table L4 in Appendix L demonstrates, the more 
additive credits earned in the first year, the more likely a degree will be completed. 

FRSHGRAD (Grade point average in the first calendar year of attendance) — Carried 
forward from the original Tool Box. 

Grade point averages were determined for the first full calendar year of postsecondary 
attendance, and were then set out in quintiles. FRSHGRAD is a dichotomous variable that 
divides the highest two quintiles from the other three. For the history of the HS&B/So used in 
the original Tool Box study, the dividing line was 2.70. For that of the NELS:88/2000 used in 
The Tool- box Revisited, the line is drawn at 2.88. The difference reflects the general trend in 
GPAs of the two cohorts (see Adehnan 2004a, table 6.1, p. 78). 

FREM (Any remedial course work in the first calendar year of attendance) — New in this 
analysis. 

FREM is a dichotomous variable marking students who took any remedial courses during the 
first calendar year of attendance. Three variations on first-year remediation were tried out in the 
process of arriving at this formulation: one that focused only on remedial reading, another on 
remedial math, and a third on all types of remediation. The dichotomous version of the latter 
was the only form that could meet the minimal statistical criteria for entrance into the stepwise 
logistic model and allow us to track any association of early remediation with degree 
completion. 
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Step 4: Financial Aid 



In general, the NCES grade-cohort longitudinal studies have been weak on financial aid 
information, where as the event-cohort Beginning Postsecondary Students longitudinal studies 
have offered a panoply of financing data. For example, if we were to ask about grants or 
scholarships received in the first year of attendance (1995-96) for the BPS:95/96-2001 by 
traditional-age students who attended a four-year college at some time, we would find the 
information indicated in table Glossary- 1 . 

We would know, too, that for those who received grants/scholarships in 1995/96, the amount 
constituted an average of 69.3 percent (s.e. = 1.0) of all aid received, and covered 33.4 percent 
(s.e. = 0.7) of the price of attendance for the 1995-96 academic year. We could easily break 
these averages out by type of institution first attended. And we would know more, as the 
BPS:95/96-2001 covers PLUS (Parent Loan for Undergraduate Students) loans, Perkins loans, 
Stafford loans (subsidized and unsubsidized), need-based aid by source, merit-aid by source, Pell 
grants, SEOG (Supplementary Educational Opportunity Grants) grants, and other types of 
financial aid. If the grade-cohort longitudinal studies such as the NELS:88/2000 had gathered 
even a third of this information, a much richer analysis would be possible — though not 
necessarily an analysis demonstrating a significant association of types, amounts, and ratios of 
financial aid to degree completion. 

Table Glossary-1. Percentage of 1995/96 beginning postsecondary students 20 years old 
and younger who attended a four- year college at any time, and 
received grants or scholarships in 1995/96, by source and average 
amount of award, and bachelor’s degree attainment rate by 2001 



Percent earning bachelor’s 
degree by 2001 



Source of grant 


Percent 

receiving 

grant 


Average 
amount of grant 


Received 
grant or 
scholarship 


Did not receive 
grant or 
scholarship 


All sources 


54.6(1.3) 


$4506.00(170.7) 


59.8(1.4) 


49.1 (1.9) 


Federal 


23.2(1.0) 


1958.60 ( 34.0) 


47.8(1.9) 


57.1 (1.4) 


State 


19.1 (1.0) 


1757.40 ( 68.8) 


58.6(2.0) 


54.0(1.4) 


Institution 


31.9(1.3) 


4312.00 (208.0) 


67.5 (1.7) 


49.0(1.5) 


Other 


16.1 (0.7) 


1529.60 ( 73.5) 


71.1 (2.0) 


51.9(1.4) 



NOTE: Standard errors are in parentheses. 

SOURCE: National Center for Education Statistics: Beginning Postsecondary Students Longitudinal Study, 
1995/96-2001, Data Analysis System (NCES 2003-173). 
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Step 5: Attendance Patterns 



FOURTRAN (Transfer from one four-year college to another) — New in this analysis. 

The algorithm for classic transfer from a community college to a four-year college was fairly 
easy to construct. But to distinguish a true four-year-to-four-year college transfer required an 
indirect route. In the final survey of NELS longitudinal study (2000), respondents who attended 
any postsecondary institution(s) were asked whether they ever "transferred credits." One cannot 
equate the yes/no answer to that question with institutional transfer. One group of four-year-to- 
four-year transfers was easy to identify: those who started in a four-year college and earned a 
bachelor’s degree from a different four-year college. The second group was defined by the 
following attendance characteristics: They started in a four-year college, attended at least two 
four-year colleges, earned more than 30 credits from four-year colleges, were not reverse 
transfers, and earned less than 20 credits from community colleges and other sub-baccalaureate 
institutions. For the most part, these students did not earn bachelor’s degrees, but they 
accumulated an average of 85.6 credits (s.e.=3.57) from four-year colleges. Of those who did 
not earn a degree, 34.6 percent (s.e. = 4.17) were still enrolled and degree-candidates in 2000. 

MULTINS (Student attended more than one postsecondary institution as an 
undergraduate) — Carried forward from Answers in the Tool Box. 

MULTINS is a dichotomous variable indicating that the student attended more than one 
institution. This is a macro-vision of otherwise multidirectional student behavior. One might 
suspect that when MULTINS, TRANSFER, and FOURTRAN are in the same logistic model, at 
least one of these variables will evidence an unacceptable degree of collinearity. But within the 
context of the universe under study in The Toolbox Revisited, the correlation between 
community college transfer (TRANSFER) and MULTINS is 0.3363 and that between four-year- 
to-four-year transfer (FOURTRAN) and MULTINS is 0.3246. These are modestly positive, but 
not so high as to indicate collinearity problems. What that may mean is that the three 
independent variables are making distinctly different statements in the context of the logistic 
regressions in which they appear. 

SUMMER (Number of credits earned during summer terms) — New in this analysis. 

Attendance and credit generation during summer terms turned out to be a strong proxy for high- 
octane persistence among community college students (Adehnan 2005a), and was, hence, 
brought into the model for the population of those who attended a four-year college at any time. 
The variable was set by three bands of the number of credits earned in this manner: more than 4, 
1-4, and 0. Some 54.8 percent (s.e. = 0.92) of all 1992 12th-graders who continued their 
education earned credits in at least one summer term; 61.1 percent (s.e. = 0.96) of those who 
earned more than 10 credits did so, as did 63.2 percent (s.e. = 1.10) of the universe on which this 
essay focuses. Given the magnitude of those percentages, it is not so much the fact of using 
summer tenns, but the volume of credit generation that should count. 

PARTTIME (Was the student’s enrollment intensity ever part-time?) — New in this analysis. 

The original Tool Box declined to confront part-time status and its effects. If one is using 
transcripts as evidence, there are a number of problems in determining which students are part- 
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time and when. This issue deserves extended attention, with support drawn from very different 
data sources (see, e.g., O’Toole, Stratton, and Wetzel 2003). Even if one standardizes credits on 
a semester metric across over 3000 institutions in the NELS:88/2000 postsecondary transcript 
files, it is often difficult to know what is considered “part-time” in term X at institution Y in 
program Z. With over 60 percent of students earning credits in summer tenns, we would be 
obligated to include such tenns in an account of enrollment intensity, and the question, “What is 
‘full-time’ in a summer tenn?” would have to be answered in each institution. In the last survey 
of the NELS:88/2000 cohort in 2000, students were asked whether they had ever “attended less 
than full time.” Despite the ambiguity of the phrasing and students’ understanding of what "less 
than full time means," student responses were accepted as the first stage of creating a variable 
indicating part-time status at some point in their careers. In their 1994 interviews, students were 
also asked about their enrollment intensity status at the first institution they attended. When they 
indicated part-time, that fact overrode any denial of part-time status in their 2000 interview. 
Proximate judgment supercedes retrospection. 

The second stage in developing this rough proxy draws on the transcripts and the notion of 
reducing credit load to part-time status. To identify students who may start with a full-time 
schedule but reduce it to part-time intensity by withdrawing from courses, we take the average 
credit load per semester for bachelor’s degree recipients — 14.2 (average annual load of 30.8 
minus the sum of summer-term credits, credits by examination, and dual enrollment credits 
divided by two) — and ask by what percentage that figure would have to fall to bring the 
student’s credit load below the threshold for full-time status (12 credits). The answer is 16.2 
percent. So for the sake of this crude proxy measure, it is assumed that students who withdrew 
from attempted credits at 16.2 percent rate or higher were — or became — part-time at some time. 
With that group added to students who indicated part-time status in the course of their 2000 
interviews, the proportion of NELS students who were part-time at some time in the 
undergraduate careers was 47.4 percent — compared with 48.2 percent of traditional-age students 
in the BPS longitudinal study of 1995/96-2001 (for an account of enrollment intensity in both 
data sets, see Appendix L, table L5). 

Step 6: Extended Postsecondary Performance 

REMPROB (Nature and intensity of remedial work) — Carried forward from Answers in the 
Tool Box. 

In both the original Tool Box and The Toolbox Revisited, REMPROB describes the nature and 
extent of a student’s remedial course work without the boundary of the first calendar year of 
attendance. For that reason, when REMPROB is tried out in the logistic narrative at the stage of 
extended postsecondary perfonnance, it replaces the freshman year remediation variable (FREM). 

REMPROB is created by an if-then-else logic that starts with remedial reading as the most 
serious of remedial problems, 75 and works its logical steps as follows: 



75 For an elaboration of REMPROB in relation to student demographic backgrounds, high school 
performance, and degree attainment, see Adelman 2004a, pp. 87-94, and Wirt, et al. (2000), indicator 14, p. 52 and 
supplemental tables (p. 152). 
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1 . Any remedial reading 

2. Two or more remedial courses (not reading, and, if math, more than two) 

3. One or two courses of remedial mathematics only 

4. One remedial course (not reading, not math, thus leaving writing, 
language arts, and general basic skills) 

5. No remedial course work 

For the sake of presentation in the logistic sequence, REMPROB is a dichotomous variable in 
which remedial reading plus more than one remedial course falls on one side of the border, and 
all other remediation (including no remediation) on the other side. In the original Tool Box, 
REMPROB was admitted into the model, but did not reach the threshold of statistical 
significance. In The Toolbox Revisited, REMPROB did not meet the minimum statistical 
standard for entry into the model when it was invoked in Step 6. 

TREND (Trend in student’s grade point average) — Modified from previous presentation in 
the original Tool Box. 

This variable describes the trend in a student’s postsecondary grade point average (GPA), and 
applies only to those students with complete undergraduate transcript records and computable 
GPAs. In its original Tool Box version, two reference points were used: end of the first calendar 
year GPA, and final undergraduate GPA. From these two markers, one could observe whether 
GPA was rising, flat, or falling, using a ratio of final GPA to first-year GPA. Ratios in the .95 to 
1.05 range were judged to indicate no change, with ratios above 1.05 indicating a rising GPA, 
and ratios below 0.95 indicating a falling GPA. For a slightly better set of markers, the 
NELS:88/2000 postsecondary transcript files thus added a variable for GPA two years after 
entry. With three points of reference, ratios are not invoked, rather relationships (greater, less 
than, or equal to) — and at three points in time. From the permutations of these relationships one 
can determine whether the trend was rising, flat/inconsistent, or falling. 

CUMMATH (Number of credits earned in college-level mathematics) — New in this analysis. 

This variable, setting out cumulative credits earned in college-level mathematics in three bands 
(more than four, 1-4, and 0) replaces the parallel variable for first-year college-level 
mathematics credits in the context of an extended postsecondary history account. The 
motivation for including this curriculum marker is the same: Even if postsecondary students do 
not concentrate their studies in science, technology, engineering, or mathematics/computer 
science (STEM), their quantitative background will be called upon (in business and the social 
sciences, in particular). Student academic momentum, it is hypothesized, will be as much 
enhanced by quantitative study on the post-matriculation side of education history as it was in 
secondary school. 

CHANMAJ (Change of major) — New in this analysis. 

If transcripts are our principal source of evidence, the only moment at which we are sure of a 
student’s major is the moment of degree award. Sometimes, the NELS:88/2000 transcripts with 
no degree indicated a major, which was verified by hand-and-eye reading by two judges only 
when the transcript carried a minimum of 15 credits and the major field indicated carried a 
minimum of 12. But that still leaves a large swath of missing accounts of students’ major. It is 
wholly possible, for example, that a student earning a B.S. in computer science changed majors 
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twice along the way, and in two different institutions. In the matter of major — and change-of- 
major — the transcript is not as credible as the student. So the last NELS survey in 2000 asked 
students whether they had changed majors at any time, but unfortunately did not ask “from what 
to what.” 

A dichotomous change-of-major variable was developed and tried out for this study. Basically, 
it took the universe of students who said they changed majors, and added to that universe 
community college transfer students who moved from general studies to a discrete major, and 
cases where the student presented two transcripts with no degrees but different majors. It also 
took student responses in 1994 to questions about their major at the first institution attended and 
that at the most recent institution attended. Where the student reported two different disciplinary 
majors, e.g. electrical engineering at the first institution and biology at the most recent 
institution, that, too, was marked as a change of major no matter what the student told 
interviewers in 2000. Unfortunately, all these steps created a collinearity problem with the 
TRANSFER variable, and CHANMAJ did not qualify for entrance into the logistic narrative. 

On the other hand, it was admitted and played a notable role in the linear account of time to 
degree (see Appendix J). 



Step 7: Final Model 

NOSTOP (Continuous enrollment) — Carried forward from Answers in the Tool Box. 

Given the extended potential undergraduate periods of 12 years (National Longitudinal Study of 
the High School Class of 1972), 1 1 years (High School and Beyond/Sophomores), or 8.5 years 
(NELS:88/2000), it makes no sense to describe stop-out behavior as one semester or its 
equivalent. In all three NCES postsecondary transcript-based grade-cohort studies, non- 
continuous enrollment was defined as more than a one semester (or its equivalent, e.g., two 
quarters) stop-out period. In the dichotomous variable, NOSTOP, the student is considered 
continuously enrolled even with one semester (or two quarters) off. From the moment of its 
introduction, NO STOP was the strongest independent variable in the logistic narrative of the 
original Tool Box. 

WRPT Ratio (Ratio of non-penalty withdrawal and no-credit repeat grades to all grades 
received) — Carried forward from the original Tool Box. 

In the original Tool Box, this variable was called the DWI (drops, withdrawals, and incompletes) 
index. DWI is not a wholly accurate acronym. What is really described is a ratio in which the 
number of courses from which the student withdrew without penalty plus those the student 
repeated is the numerator, and the total number of courses in which the student enrolled is the 
denominator. The ratio counts course attempts, not credits. Withdrawals without penalty are not 
the same as courses “dropped” within set periods that most colleges and community colleges 
mark for “drop-and-add.” Courses “dropped” are not included in the ratio. Nothing here 
involves penalty grades. All cases at issue are noncredit grades. The variable WRPT ratio is 
dichotomous: On one side of the dividing line are students who withdrew from or repeated 20 
percent or more of all courses in which they enrolled. 
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